2024-09-13T17:31:58.265 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-13T17:31:58.286 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T17:31:58.400 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903920 branch: wip-rishabh-testing-20240910.100730-reef description: fs/upgrade/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/quincy 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/new_ops/{newops}}} email: null first_in_suite: false job_id: '7903920' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'install ceph/quincy latest ' name: rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240910.100730-reef ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 mon warn on pool no app: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240910.100730-reef sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.b - mon.c - mgr.x - mds.a - osd.0 - osd.1 - osd.2 - osd.3 - - client.0 seed: 6439 sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 sleep_before_teardown: 0 subset: 964/1024 suite: fs suite_branch: wip-rishabh-testing-20240910.100730-reef suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 targets: smithi102.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPCxH4XlJVa0qTASLbCPXm4S2jgGhwiQoc+YoAYL+BlUWqlpkFLp/koBmYbttNW3l/qBHSrAaEv41semcbjhuSI= smithi171.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAJmIrHYCHfQDZnEHx3qzeAFlpSvdSb2q7Wo2Y9GFRunr/lWaf62giCTPxT013ljyzUTTqESTllGhoFUI+yCY2I= tasks: - install: branch: quincy exclude_packages: - cephadm - ceph-mgr-cephadm - ceph-immutable-object-cache - python3-rados - python3-rgw - python3-rbd - python3-cephfs - ceph-volume extra_packages: - python-rados - python-rgw - python-rbd - python-cephfs extra_system_packages: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - print: '**** done installing quincy' - ceph: conf: global: mon warn on pool no app: false ms bind msgr2: false log-ignorelist: - overall HEALTH_ - \(FS_ - \(MDS_ - \(OSD_ - \(MON_DOWN\) - \(CACHE_POOL_ - \(POOL_ - \(MGR_DOWN\) - \(PG_ - \(SMALLER_PGP_NUM\) - Monitor daemon marked osd - Behind on trimming - Manager daemon - exec: osd.0: - ceph osd set-require-min-compat-client quincy - print: '**** done ceph' - ceph-fuse: null - ceph-fuse: client.0: mounted: false - print: '**** done unmount client.0' - install.upgrade: client.0: null - print: '**** done install.upgrade on client.0' - ceph.healthy: null - ceph-fuse: client.0: null - print: '**** done remount client.0' - cephfs_test_runner: modules: - tasks.cephfs.test_newops teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end - if not is_fuse() then reject() end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-13_14:42:16 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-13T17:31:58.400 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/qa; will attempt to use it 2024-09-13T17:31:58.401 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/qa/tasks 2024-09-13T17:31:58.401 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-13T17:31:58.403 INFO:teuthology.task.internal:Checking packages... 2024-09-13T17:31:58.432 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3' 2024-09-13T17:31:58.432 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T17:31:58.432 INFO:teuthology.packaging:ref: None 2024-09-13T17:31:58.432 INFO:teuthology.packaging:tag: None 2024-09-13T17:31:58.432 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240910.100730-reef 2024-09-13T17:31:58.432 INFO:teuthology.packaging:sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:31:58.432 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240910.100730-reef 2024-09-13T17:31:58.586 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-860.g6c6b083f 2024-09-13T17:31:58.588 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-13T17:31:58.594 INFO:teuthology.task.internal:no buildpackages task found 2024-09-13T17:31:58.594 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-13T17:31:58.666 INFO:teuthology.task.internal:Saving configuration 2024-09-13T17:31:58.683 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-13T17:31:58.729 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-13T17:31:58.759 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi102.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903920', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 17:26:44.492461', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPCxH4XlJVa0qTASLbCPXm4S2jgGhwiQoc+YoAYL+BlUWqlpkFLp/koBmYbttNW3l/qBHSrAaEv41semcbjhuSI='} 2024-09-13T17:31:58.785 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi171.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903920', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 17:26:44.494330', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAJmIrHYCHfQDZnEHx3qzeAFlpSvdSb2q7Wo2Y9GFRunr/lWaf62giCTPxT013ljyzUTTqESTllGhoFUI+yCY2I='} 2024-09-13T17:31:58.785 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-13T17:31:58.791 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2024-09-13T17:31:58.791 INFO:teuthology.task.internal:roles: ubuntu@smithi171.front.sepia.ceph.com - ['client.0'] 2024-09-13T17:31:58.791 INFO:teuthology.run_tasks:Running task console_log... 2024-09-13T17:31:58.868 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fda9ec47be0>, signals=[15]) 2024-09-13T17:31:58.868 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-13T17:31:58.876 INFO:teuthology.task.internal:Opening connections... 2024-09-13T17:31:58.877 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi102.front.sepia.ceph.com 2024-09-13T17:31:58.878 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:31:59.027 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi171.front.sepia.ceph.com 2024-09-13T17:31:59.028 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:31:59.111 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-13T17:31:59.120 DEBUG:teuthology.orchestra.run.smithi102:> uname -m 2024-09-13T17:31:59.138 INFO:teuthology.orchestra.run.smithi102.stdout:x86_64 2024-09-13T17:31:59.139 DEBUG:teuthology.orchestra.run.smithi102:> cat /etc/os-release 2024-09-13T17:31:59.196 INFO:teuthology.orchestra.run.smithi102.stdout:NAME="CentOS Stream" 2024-09-13T17:31:59.196 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION="9" 2024-09-13T17:31:59.196 INFO:teuthology.orchestra.run.smithi102.stdout:ID="centos" 2024-09-13T17:31:59.196 INFO:teuthology.orchestra.run.smithi102.stdout:ID_LIKE="rhel fedora" 2024-09-13T17:31:59.196 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION_ID="9" 2024-09-13T17:31:59.197 INFO:teuthology.orchestra.run.smithi102.stdout:PLATFORM_ID="platform:el9" 2024-09-13T17:31:59.197 INFO:teuthology.orchestra.run.smithi102.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-13T17:31:59.197 INFO:teuthology.orchestra.run.smithi102.stdout:ANSI_COLOR="0;31" 2024-09-13T17:31:59.197 INFO:teuthology.orchestra.run.smithi102.stdout:LOGO="fedora-logo-icon" 2024-09-13T17:31:59.197 INFO:teuthology.orchestra.run.smithi102.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-13T17:31:59.197 INFO:teuthology.orchestra.run.smithi102.stdout:HOME_URL="https://centos.org/" 2024-09-13T17:31:59.197 INFO:teuthology.orchestra.run.smithi102.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-13T17:31:59.197 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-13T17:31:59.197 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-13T17:31:59.198 INFO:teuthology.lock.ops:Updating smithi102.front.sepia.ceph.com on lock server 2024-09-13T17:31:59.229 DEBUG:teuthology.orchestra.run.smithi171:> uname -m 2024-09-13T17:31:59.248 INFO:teuthology.orchestra.run.smithi171.stdout:x86_64 2024-09-13T17:31:59.248 DEBUG:teuthology.orchestra.run.smithi171:> cat /etc/os-release 2024-09-13T17:31:59.306 INFO:teuthology.orchestra.run.smithi171.stdout:NAME="CentOS Stream" 2024-09-13T17:31:59.306 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION="9" 2024-09-13T17:31:59.306 INFO:teuthology.orchestra.run.smithi171.stdout:ID="centos" 2024-09-13T17:31:59.306 INFO:teuthology.orchestra.run.smithi171.stdout:ID_LIKE="rhel fedora" 2024-09-13T17:31:59.307 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION_ID="9" 2024-09-13T17:31:59.307 INFO:teuthology.orchestra.run.smithi171.stdout:PLATFORM_ID="platform:el9" 2024-09-13T17:31:59.307 INFO:teuthology.orchestra.run.smithi171.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-13T17:31:59.307 INFO:teuthology.orchestra.run.smithi171.stdout:ANSI_COLOR="0;31" 2024-09-13T17:31:59.307 INFO:teuthology.orchestra.run.smithi171.stdout:LOGO="fedora-logo-icon" 2024-09-13T17:31:59.307 INFO:teuthology.orchestra.run.smithi171.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-13T17:31:59.307 INFO:teuthology.orchestra.run.smithi171.stdout:HOME_URL="https://centos.org/" 2024-09-13T17:31:59.307 INFO:teuthology.orchestra.run.smithi171.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-13T17:31:59.307 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-13T17:31:59.307 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-13T17:31:59.307 INFO:teuthology.lock.ops:Updating smithi171.front.sepia.ceph.com on lock server 2024-09-13T17:31:59.336 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-13T17:31:59.345 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-13T17:31:59.386 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-13T17:31:59.386 DEBUG:teuthology.orchestra.run.smithi102:> test '!' -e /home/ubuntu/cephtest 2024-09-13T17:31:59.390 DEBUG:teuthology.orchestra.run.smithi171:> test '!' -e /home/ubuntu/cephtest 2024-09-13T17:31:59.408 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-13T17:31:59.438 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-13T17:31:59.438 DEBUG:teuthology.orchestra.run.smithi102:> test -z $(ls -A /var/lib/ceph) 2024-09-13T17:31:59.446 DEBUG:teuthology.orchestra.run.smithi171:> test -z $(ls -A /var/lib/ceph) 2024-09-13T17:31:59.566 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-13T17:31:59.631 INFO:teuthology.run_tasks:Running task kernel... 2024-09-13T17:31:59.647 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-13T17:31:59.647 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'client.0': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-13T17:31:59.647 DEBUG:teuthology.orchestra.run.smithi102:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T17:31:59.648 DEBUG:teuthology.orchestra.run.smithi171:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T17:31:59.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:31:59.664 DEBUG:teuthology.orchestra.run.smithi102:> uname -r 2024-09-13T17:31:59.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:31:59.666 DEBUG:teuthology.orchestra.run.smithi171:> uname -r 2024-09-13T17:31:59.722 INFO:teuthology.orchestra.run.smithi102.stdout:5.14.0-503.el9.x86_64 2024-09-13T17:31:59.722 INFO:teuthology.task.kernel:Running kernel on smithi102: 5.14.0-503.el9.x86_64 2024-09-13T17:31:59.722 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum install -y kernel 2024-09-13T17:31:59.723 INFO:teuthology.orchestra.run.smithi171.stdout:5.14.0-503.el9.x86_64 2024-09-13T17:31:59.724 INFO:teuthology.task.kernel:Running kernel on smithi171: 5.14.0-503.el9.x86_64 2024-09-13T17:31:59.724 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum install -y kernel 2024-09-13T17:32:02.110 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.2 MB 00:00 2024-09-13T17:32:03.157 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.2 MB 00:01 2024-09-13T17:32:03.785 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - AppStream 36 MB/s | 20 MB 00:00 2024-09-13T17:32:06.991 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - AppStream 7.5 MB/s | 20 MB 00:02 2024-09-13T17:32:09.061 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.5 MB 00:01 2024-09-13T17:32:10.595 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 19 kB 00:00 2024-09-13T17:32:12.066 INFO:teuthology.orchestra.run.smithi102.stdout:Extra Packages for Enterprise Linux 17 MB/s | 23 MB 00:01 2024-09-13T17:32:12.900 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - CRB 3.1 MB/s | 6.5 MB 00:02 2024-09-13T17:32:14.526 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - Extras packages 47 kB/s | 19 kB 00:00 2024-09-13T17:32:15.939 INFO:teuthology.orchestra.run.smithi171.stdout:Extra Packages for Enterprise Linux 18 MB/s | 23 MB 00:01 2024-09-13T17:32:17.447 INFO:teuthology.orchestra.run.smithi102.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-13T17:32:19.470 INFO:teuthology.orchestra.run.smithi102.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-13T17:32:19.470 INFO:teuthology.orchestra.run.smithi102.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-13T17:32:19.565 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-13T17:32:19.572 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-13T17:32:19.572 INFO:teuthology.orchestra.run.smithi102.stdout: Package Architecture Version Repository Size 2024-09-13T17:32:19.572 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-13T17:32:19.572 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-13T17:32:19.572 INFO:teuthology.orchestra.run.smithi102.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-13T17:32:19.572 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-13T17:32:19.572 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-13T17:32:19.572 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-13T17:32:19.572 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-13T17:32:19.572 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:32:19.572 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-13T17:32:19.573 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-13T17:32:19.573 INFO:teuthology.orchestra.run.smithi102.stdout:Install 4 Packages 2024-09-13T17:32:19.573 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:32:19.573 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 79 M 2024-09-13T17:32:19.573 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 126 M 2024-09-13T17:32:19.573 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-13T17:32:20.165 INFO:teuthology.orchestra.run.smithi102.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 42 kB/s | 23 kB 00:00 2024-09-13T17:32:21.049 INFO:teuthology.orchestra.run.smithi102.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-13T17:32:21.352 INFO:teuthology.orchestra.run.smithi171.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-13T17:32:21.716 INFO:teuthology.orchestra.run.smithi102.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 17 MB/s | 35 MB 00:02 2024-09-13T17:32:22.399 INFO:teuthology.orchestra.run.smithi102.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 13 MB/s | 29 MB 00:02 2024-09-13T17:32:22.399 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-13T17:32:22.400 INFO:teuthology.orchestra.run.smithi102.stdout:Total 28 MB/s | 79 MB 00:02 2024-09-13T17:32:23.085 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-13T17:32:23.426 INFO:teuthology.orchestra.run.smithi171.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-13T17:32:23.426 INFO:teuthology.orchestra.run.smithi171.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-13T17:32:23.427 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-13T17:32:23.428 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-13T17:32:23.519 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-13T17:32:23.525 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-13T17:32:23.525 INFO:teuthology.orchestra.run.smithi171.stdout: Package Architecture Version Repository Size 2024-09-13T17:32:23.525 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-13T17:32:23.525 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-13T17:32:23.526 INFO:teuthology.orchestra.run.smithi171.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-13T17:32:23.526 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-09-13T17:32:23.526 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-13T17:32:23.526 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-13T17:32:23.526 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-13T17:32:23.526 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:32:23.526 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-13T17:32:23.526 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-09-13T17:32:23.526 INFO:teuthology.orchestra.run.smithi171.stdout:Install 4 Packages 2024-09-13T17:32:23.526 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:32:23.527 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 79 M 2024-09-13T17:32:23.527 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 126 M 2024-09-13T17:32:23.527 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-13T17:32:23.696 INFO:teuthology.orchestra.run.smithi171.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 186 kB/s | 23 kB 00:00 2024-09-13T17:32:24.392 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-13T17:32:24.393 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-13T17:32:26.838 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-13T17:32:27.492 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-13T17:32:27.583 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T17:32:29.837 INFO:teuthology.orchestra.run.smithi171.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 2.5 MB/s | 16 MB 00:06 2024-09-13T17:32:30.367 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T17:32:30.513 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T17:32:30.612 INFO:teuthology.orchestra.run.smithi171.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 4.9 MB/s | 35 MB 00:07 2024-09-13T17:32:31.379 INFO:teuthology.orchestra.run.smithi171.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 3.7 MB/s | 29 MB 00:07 2024-09-13T17:32:31.379 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-13T17:32:31.379 INFO:teuthology.orchestra.run.smithi171.stdout:Total 10 MB/s | 79 MB 00:07 2024-09-13T17:32:32.068 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-13T17:32:32.434 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-13T17:32:32.434 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-13T17:32:33.394 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-13T17:32:33.394 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-13T17:32:35.349 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T17:32:35.396 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:32:35.868 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-13T17:32:36.531 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-13T17:32:36.622 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T17:32:39.429 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T17:32:39.557 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T17:32:40.260 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:32:44.413 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T17:32:44.448 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:32:49.320 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:33:18.133 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:33:18.173 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:33:18.476 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:33:18.476 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-13T17:33:18.477 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T17:33:18.477 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T17:33:19.337 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:33:19.341 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:33:19.341 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-13T17:33:19.341 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-13T17:33:19.341 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-13T17:33:19.341 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-13T17:33:19.341 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-13T17:33:19.341 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:33:19.341 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-13T17:33:19.686 DEBUG:teuthology.orchestra.run.smithi102:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T17:33:19.762 INFO:teuthology.orchestra.run.smithi102.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T17:33:19.762 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T17:33:19.762 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-13T17:33:19.763 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T17:33:19.763 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T17:33:19.763 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-13T17:33:19.763 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi102.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T17:33:19.763 DEBUG:teuthology.orchestra.run.smithi102:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T17:33:19.822 INFO:teuthology.orchestra.run.smithi102.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T17:33:19.822 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T17:33:19.822 DEBUG:teuthology.orchestra.run.smithi102:> sudo rpm -qi grub2-tools 2024-09-13T17:33:19.853 INFO:teuthology.orchestra.run.smithi102.stdout:Name : grub2-tools 2024-09-13T17:33:19.853 INFO:teuthology.orchestra.run.smithi102.stdout:Epoch : 1 2024-09-13T17:33:19.854 INFO:teuthology.orchestra.run.smithi102.stdout:Version : 2.06 2024-09-13T17:33:19.854 INFO:teuthology.orchestra.run.smithi102.stdout:Release : 82.el9 2024-09-13T17:33:19.854 INFO:teuthology.orchestra.run.smithi102.stdout:Architecture: x86_64 2024-09-13T17:33:19.854 INFO:teuthology.orchestra.run.smithi102.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-13T17:33:19.854 INFO:teuthology.orchestra.run.smithi102.stdout:Group : Unspecified 2024-09-13T17:33:19.854 INFO:teuthology.orchestra.run.smithi102.stdout:Size : 8274670 2024-09-13T17:33:19.854 INFO:teuthology.orchestra.run.smithi102.stdout:License : GPLv3+ 2024-09-13T17:33:19.854 INFO:teuthology.orchestra.run.smithi102.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-13T17:33:19.854 INFO:teuthology.orchestra.run.smithi102.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-13T17:33:19.854 INFO:teuthology.orchestra.run.smithi102.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-13T17:33:19.854 INFO:teuthology.orchestra.run.smithi102.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-13T17:33:19.854 INFO:teuthology.orchestra.run.smithi102.stdout:Packager : builder@centos.org 2024-09-13T17:33:19.854 INFO:teuthology.orchestra.run.smithi102.stdout:Vendor : CentOS 2024-09-13T17:33:19.854 INFO:teuthology.orchestra.run.smithi102.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-13T17:33:19.855 INFO:teuthology.orchestra.run.smithi102.stdout:Summary : Support tools for GRUB. 2024-09-13T17:33:19.855 INFO:teuthology.orchestra.run.smithi102.stdout:Description : 2024-09-13T17:33:19.855 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:33:19.855 INFO:teuthology.orchestra.run.smithi102.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-13T17:33:19.855 INFO:teuthology.orchestra.run.smithi102.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-13T17:33:19.855 INFO:teuthology.orchestra.run.smithi102.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-13T17:33:19.855 INFO:teuthology.orchestra.run.smithi102.stdout:hardware devices. 2024-09-13T17:33:19.855 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:33:19.855 INFO:teuthology.orchestra.run.smithi102.stdout:This subpackage provides tools for support of all platforms. 2024-09-13T17:33:19.856 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-13T17:33:19.856 INFO:teuthology.task.kernel:Updating grub on smithi102 to boot 5.14.0-505.el9.x86_64 2024-09-13T17:33:19.856 DEBUG:teuthology.orchestra.run.smithi102:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-13T17:33:20.511 INFO:teuthology.orchestra.run.smithi102.stderr:Generating grub configuration file ... 2024-09-13T17:33:21.815 INFO:teuthology.orchestra.run.smithi102.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-13T17:33:21.835 INFO:teuthology.orchestra.run.smithi102.stderr:done 2024-09-13T17:33:21.837 DEBUG:teuthology.orchestra.run.smithi102:> mktemp 2024-09-13T17:33:21.854 INFO:teuthology.orchestra.run.smithi102.stdout:/tmp/tmp.higxlBWbhX 2024-09-13T17:33:21.854 DEBUG:teuthology.orchestra.run.smithi102:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.higxlBWbhX 2024-09-13T17:33:21.921 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 0666 /tmp/tmp.higxlBWbhX 2024-09-13T17:33:22.073 DEBUG:teuthology.orchestra.remote:smithi102:/tmp/tmp.higxlBWbhX is 6KB 2024-09-13T17:33:22.122 DEBUG:teuthology.orchestra.run.smithi102:> rm -fr /tmp/tmp.higxlBWbhX 2024-09-13T17:33:22.138 DEBUG:teuthology.orchestra.run.smithi102:> sudo /bin/ls /boot/loader/entries || true 2024-09-13T17:33:22.200 INFO:teuthology.orchestra.run.smithi102.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-13T17:33:22.200 INFO:teuthology.orchestra.run.smithi102.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-13T17:33:22.200 INFO:teuthology.orchestra.run.smithi102.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-13T17:33:22.200 INFO:teuthology.orchestra.run.smithi102.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-13T17:33:22.205 DEBUG:teuthology.orchestra.run.smithi102:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-13T17:33:22.357 DEBUG:teuthology.orchestra.run.smithi102:> sudo shutdown -r now 2024-09-13T17:33:27.275 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:33:27.320 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:33:27.653 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:33:27.653 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-13T17:33:27.653 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T17:33:27.653 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T17:33:28.526 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T17:33:28.526 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:33:28.526 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-13T17:33:28.527 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-13T17:33:28.527 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-13T17:33:28.527 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-13T17:33:28.527 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-13T17:33:28.527 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:33:28.527 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-13T17:33:28.873 DEBUG:teuthology.orchestra.run.smithi171:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T17:33:28.955 INFO:teuthology.orchestra.run.smithi171.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T17:33:28.955 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T17:33:28.955 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-13T17:33:28.955 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T17:33:28.956 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T17:33:28.956 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-13T17:33:28.956 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi171.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T17:33:28.956 DEBUG:teuthology.orchestra.run.smithi171:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T17:33:29.015 INFO:teuthology.orchestra.run.smithi171.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T17:33:29.015 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T17:33:29.015 DEBUG:teuthology.orchestra.run.smithi171:> sudo rpm -qi grub2-tools 2024-09-13T17:33:29.048 INFO:teuthology.orchestra.run.smithi171.stdout:Name : grub2-tools 2024-09-13T17:33:29.048 INFO:teuthology.orchestra.run.smithi171.stdout:Epoch : 1 2024-09-13T17:33:29.048 INFO:teuthology.orchestra.run.smithi171.stdout:Version : 2.06 2024-09-13T17:33:29.048 INFO:teuthology.orchestra.run.smithi171.stdout:Release : 82.el9 2024-09-13T17:33:29.048 INFO:teuthology.orchestra.run.smithi171.stdout:Architecture: x86_64 2024-09-13T17:33:29.048 INFO:teuthology.orchestra.run.smithi171.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-13T17:33:29.048 INFO:teuthology.orchestra.run.smithi171.stdout:Group : Unspecified 2024-09-13T17:33:29.048 INFO:teuthology.orchestra.run.smithi171.stdout:Size : 8274670 2024-09-13T17:33:29.048 INFO:teuthology.orchestra.run.smithi171.stdout:License : GPLv3+ 2024-09-13T17:33:29.048 INFO:teuthology.orchestra.run.smithi171.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-13T17:33:29.048 INFO:teuthology.orchestra.run.smithi171.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-13T17:33:29.048 INFO:teuthology.orchestra.run.smithi171.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-13T17:33:29.049 INFO:teuthology.orchestra.run.smithi171.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-13T17:33:29.049 INFO:teuthology.orchestra.run.smithi171.stdout:Packager : builder@centos.org 2024-09-13T17:33:29.049 INFO:teuthology.orchestra.run.smithi171.stdout:Vendor : CentOS 2024-09-13T17:33:29.049 INFO:teuthology.orchestra.run.smithi171.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-13T17:33:29.049 INFO:teuthology.orchestra.run.smithi171.stdout:Summary : Support tools for GRUB. 2024-09-13T17:33:29.049 INFO:teuthology.orchestra.run.smithi171.stdout:Description : 2024-09-13T17:33:29.049 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:33:29.049 INFO:teuthology.orchestra.run.smithi171.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-13T17:33:29.049 INFO:teuthology.orchestra.run.smithi171.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-13T17:33:29.049 INFO:teuthology.orchestra.run.smithi171.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-13T17:33:29.049 INFO:teuthology.orchestra.run.smithi171.stdout:hardware devices. 2024-09-13T17:33:29.049 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:33:29.049 INFO:teuthology.orchestra.run.smithi171.stdout:This subpackage provides tools for support of all platforms. 2024-09-13T17:33:29.050 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-13T17:33:29.050 INFO:teuthology.task.kernel:Updating grub on smithi171 to boot 5.14.0-505.el9.x86_64 2024-09-13T17:33:29.050 DEBUG:teuthology.orchestra.run.smithi171:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-13T17:33:29.676 INFO:teuthology.orchestra.run.smithi171.stderr:Generating grub configuration file ... 2024-09-13T17:33:31.059 INFO:teuthology.orchestra.run.smithi171.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-13T17:33:31.086 INFO:teuthology.orchestra.run.smithi171.stderr:done 2024-09-13T17:33:31.088 DEBUG:teuthology.orchestra.run.smithi171:> mktemp 2024-09-13T17:33:31.104 INFO:teuthology.orchestra.run.smithi171.stdout:/tmp/tmp.FqvaOvLqJm 2024-09-13T17:33:31.104 DEBUG:teuthology.orchestra.run.smithi171:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.FqvaOvLqJm 2024-09-13T17:33:31.171 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 0666 /tmp/tmp.FqvaOvLqJm 2024-09-13T17:33:31.307 DEBUG:teuthology.orchestra.remote:smithi171:/tmp/tmp.FqvaOvLqJm is 6KB 2024-09-13T17:33:31.355 DEBUG:teuthology.orchestra.run.smithi171:> rm -fr /tmp/tmp.FqvaOvLqJm 2024-09-13T17:33:31.371 DEBUG:teuthology.orchestra.run.smithi171:> sudo /bin/ls /boot/loader/entries || true 2024-09-13T17:33:31.433 INFO:teuthology.orchestra.run.smithi171.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-13T17:33:31.434 INFO:teuthology.orchestra.run.smithi171.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-13T17:33:31.434 INFO:teuthology.orchestra.run.smithi171.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-13T17:33:31.434 INFO:teuthology.orchestra.run.smithi171.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-13T17:33:31.435 DEBUG:teuthology.orchestra.run.smithi171:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-13T17:33:31.597 DEBUG:teuthology.orchestra.run.smithi171:> sudo shutdown -r now 2024-09-13T17:33:52.377 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-13T17:33:52.377 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-09-13T17:33:52.378 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:34:01.609 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-13T17:34:01.609 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-13T17:34:01.609 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:34:19.969 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.171 2024-09-13T17:34:28.977 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-13T17:34:28.978 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:34:32.065 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.171 2024-09-13T17:34:44.077 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-13T17:34:44.077 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:34:52.390 DEBUG:teuthology.orchestra.remote:timed out 2024-09-13T17:35:01.399 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-09-13T17:35:01.400 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:35:01.715 DEBUG:teuthology.orchestra.run.smithi102:> true 2024-09-13T17:35:02.091 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-09-13T17:35:02.091 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-13T17:35:02.091 DEBUG:teuthology.orchestra.run.smithi102:> uname -r 2024-09-13T17:35:02.106 INFO:teuthology.orchestra.run.smithi102.stdout:5.14.0-505.el9.x86_64 2024-09-13T17:35:02.106 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-13T17:35:02.106 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T17:35:02.106 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-13T17:35:03.107 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-13T17:35:03.107 DEBUG:teuthology.orchestra.run.smithi102:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T17:35:03.185 INFO:teuthology.orchestra.run.smithi102.stdout:ttyS1 2024-09-13T17:35:03.210 DEBUG:teuthology.parallel:result is None 2024-09-13T17:35:44.105 DEBUG:teuthology.orchestra.remote:timed out 2024-09-13T17:35:59.106 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-13T17:35:59.107 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:35:59.452 DEBUG:teuthology.orchestra.run.smithi171:> true 2024-09-13T17:35:59.782 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-13T17:35:59.782 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-13T17:35:59.782 DEBUG:teuthology.orchestra.run.smithi171:> uname -r 2024-09-13T17:35:59.798 INFO:teuthology.orchestra.run.smithi171.stdout:5.14.0-505.el9.x86_64 2024-09-13T17:35:59.799 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-13T17:35:59.799 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T17:35:59.799 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-13T17:36:00.800 INFO:teuthology.task.kernel:Enabling kdb on client.0... 2024-09-13T17:36:00.800 DEBUG:teuthology.orchestra.run.smithi171:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T17:36:00.892 INFO:teuthology.orchestra.run.smithi171.stdout:ttyS1 2024-09-13T17:36:00.912 DEBUG:teuthology.parallel:result is None 2024-09-13T17:36:00.912 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-13T17:36:00.922 INFO:teuthology.task.internal:Creating test directory... 2024-09-13T17:36:00.922 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T17:36:00.925 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T17:36:00.943 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-13T17:36:00.957 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-13T17:36:00.971 INFO:teuthology.task.internal:Creating archive directory... 2024-09-13T17:36:00.971 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T17:36:00.984 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T17:36:01.066 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-13T17:36:01.077 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-13T17:36:01.077 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T17:36:01.080 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T17:36:01.116 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T17:36:01.116 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T17:36:01.127 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T17:36:01.127 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T17:36:01.129 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-13T17:36:01.140 INFO:teuthology.task.internal:Configuring sudo... 2024-09-13T17:36:01.140 DEBUG:teuthology.orchestra.run.smithi102:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T17:36:01.172 DEBUG:teuthology.orchestra.run.smithi171:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T17:36:01.199 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-13T17:36:01.212 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-13T17:36:01.212 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T17:36:01.240 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T17:36:01.256 DEBUG:teuthology.orchestra.run.smithi102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T17:36:01.344 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T17:36:01.387 DEBUG:teuthology.orchestra.run.smithi102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T17:36:01.478 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T17:36:01.511 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-13T17:36:01.511 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T17:36:01.576 DEBUG:teuthology.orchestra.run.smithi171:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T17:36:01.628 DEBUG:teuthology.orchestra.run.smithi171:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T17:36:01.678 DEBUG:teuthology.orchestra.run.smithi171:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T17:36:01.772 DEBUG:teuthology.orchestra.run.smithi171:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T17:36:01.808 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T17:36:01.808 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T17:36:01.875 DEBUG:teuthology.orchestra.run.smithi102:> sudo service rsyslog restart 2024-09-13T17:36:01.878 DEBUG:teuthology.orchestra.run.smithi171:> sudo service rsyslog restart 2024-09-13T17:36:01.920 INFO:teuthology.orchestra.run.smithi102.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T17:36:01.969 INFO:teuthology.orchestra.run.smithi171.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T17:36:02.438 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-13T17:36:02.446 INFO:teuthology.task.internal:Starting timer... 2024-09-13T17:36:02.446 INFO:teuthology.run_tasks:Running task pcp... 2024-09-13T17:36:02.458 INFO:teuthology.run_tasks:Running task selinux... 2024-09-13T17:36:02.467 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-13T17:36:02.468 DEBUG:teuthology.orchestra.run.smithi102:> sudo service auditd rotate 2024-09-13T17:36:02.522 INFO:teuthology.orchestra.run.smithi102.stdout:Rotating logs: 2024-09-13T17:36:02.524 DEBUG:teuthology.orchestra.run.smithi171:> sudo service auditd rotate 2024-09-13T17:36:02.573 INFO:teuthology.orchestra.run.smithi171.stdout:Rotating logs: 2024-09-13T17:36:02.575 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-13T17:36:02.576 DEBUG:teuthology.orchestra.run.smithi102:> /usr/sbin/getenforce 2024-09-13T17:36:02.605 INFO:teuthology.orchestra.run.smithi102.stdout:Permissive 2024-09-13T17:36:02.605 DEBUG:teuthology.orchestra.run.smithi171:> /usr/sbin/getenforce 2024-09-13T17:36:02.636 INFO:teuthology.orchestra.run.smithi171.stdout:Permissive 2024-09-13T17:36:02.636 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi102.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi171.front.sepia.ceph.com': 'permissive'} 2024-09-13T17:36:02.637 DEBUG:teuthology.orchestra.run.smithi102:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-13T17:36:02.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:36:02.673 DEBUG:teuthology.orchestra.run.smithi171:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-13T17:36:02.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:36:02.703 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-13T17:36:02.703 DEBUG:teuthology.orchestra.run.smithi102:> sudo /usr/sbin/setenforce permissive 2024-09-13T17:36:02.746 DEBUG:teuthology.orchestra.run.smithi171:> sudo /usr/sbin/setenforce permissive 2024-09-13T17:36:02.776 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-13T17:36:02.785 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-13T17:36:02.803 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-13T17:36:02.804 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi102.front.sepia.ceph.com,smithi171.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-13T17:39:48.254 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi102.front.sepia.ceph.com'), Remote(name='ubuntu@smithi171.front.sepia.ceph.com')] 2024-09-13T17:39:48.256 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-09-13T17:39:48.258 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:39:48.343 DEBUG:teuthology.orchestra.run.smithi102:> true 2024-09-13T17:39:48.428 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-09-13T17:39:48.429 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-13T17:39:48.429 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:39:48.512 DEBUG:teuthology.orchestra.run.smithi171:> true 2024-09-13T17:39:48.589 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-13T17:39:48.589 INFO:teuthology.run_tasks:Running task clock... 2024-09-13T17:39:48.598 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-13T17:39:48.598 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T17:39:48.599 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T17:39:48.602 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T17:39:48.603 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T17:39:48.629 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-13T17:39:48.643 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-13T17:39:48.658 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-13T17:39:48.667 INFO:teuthology.orchestra.run.smithi102.stderr:sudo: ntpd: command not found 2024-09-13T17:39:48.672 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-13T17:39:48.678 INFO:teuthology.orchestra.run.smithi102.stdout:506 Cannot talk to daemon 2024-09-13T17:39:48.693 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-13T17:39:48.696 INFO:teuthology.orchestra.run.smithi171.stderr:sudo: ntpd: command not found 2024-09-13T17:39:48.707 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-13T17:39:48.708 INFO:teuthology.orchestra.run.smithi171.stdout:506 Cannot talk to daemon 2024-09-13T17:39:48.721 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-13T17:39:48.736 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-13T17:39:48.761 INFO:teuthology.orchestra.run.smithi102.stderr:bash: line 1: ntpq: command not found 2024-09-13T17:39:48.765 INFO:teuthology.orchestra.run.smithi102.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T17:39:48.765 INFO:teuthology.orchestra.run.smithi102.stdout:=============================================================================== 2024-09-13T17:39:48.765 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:39:48.765 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:39:48.766 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:39:48.766 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:39:48.791 INFO:teuthology.orchestra.run.smithi171.stderr:bash: line 1: ntpq: command not found 2024-09-13T17:39:48.796 INFO:teuthology.orchestra.run.smithi171.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T17:39:48.796 INFO:teuthology.orchestra.run.smithi171.stdout:=============================================================================== 2024-09-13T17:39:48.796 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:39:48.796 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:39:48.797 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:39:48.797 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:39:48.797 INFO:teuthology.run_tasks:Running task install... 2024-09-13T17:39:48.806 DEBUG:teuthology.task.install:project ceph 2024-09-13T17:39:48.806 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}} 2024-09-13T17:39:48.806 DEBUG:teuthology.task.install:config {'branch': 'quincy', 'exclude_packages': ['cephadm', 'ceph-mgr-cephadm', 'ceph-immutable-object-cache', 'python3-rados', 'python3-rgw', 'python3-rbd', 'python3-cephfs', 'ceph-volume'], 'extra_packages': ['python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'], 'extra_system_packages': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux'], 'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'} 2024-09-13T17:39:48.806 INFO:teuthology.task.install:Using flavor: default 2024-09-13T17:39:48.813 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-13T17:39:48.813 INFO:teuthology.task.install:extra packages: ['python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'] 2024-09-13T17:39:48.813 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['cephadm', 'ceph-mgr-cephadm', 'ceph-immutable-object-cache', 'python3-rados', 'python3-rgw', 'python3-rbd', 'python3-cephfs', 'ceph-volume'], 'extra_packages': ['python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'], 'extra_system_packages': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux'], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'tag': None, 'wait_for_package': False} 2024-09-13T17:39:48.814 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T17:39:48.814 INFO:teuthology.packaging:ref: None 2024-09-13T17:39:48.814 INFO:teuthology.packaging:tag: None 2024-09-13T17:39:48.814 INFO:teuthology.packaging:branch: quincy 2024-09-13T17:39:48.814 INFO:teuthology.packaging:sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:39:48.814 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-09-13T17:39:48.817 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['cephadm', 'ceph-mgr-cephadm', 'ceph-immutable-object-cache', 'python3-rados', 'python3-rgw', 'python3-rbd', 'python3-cephfs', 'ceph-volume'], 'extra_packages': ['python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'], 'extra_system_packages': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux'], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'tag': None, 'wait_for_package': False} 2024-09-13T17:39:48.817 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T17:39:48.817 INFO:teuthology.packaging:ref: None 2024-09-13T17:39:48.817 INFO:teuthology.packaging:tag: None 2024-09-13T17:39:48.817 INFO:teuthology.packaging:branch: quincy 2024-09-13T17:39:48.817 INFO:teuthology.packaging:sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:39:48.817 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-09-13T17:39:49.034 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/bbdadfa31f8fb9afaa389358d138cac49bd69e56/centos/9/flavors/default/ 2024-09-13T17:39:49.034 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1627.gbbdadfa3 2024-09-13T17:39:49.067 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/bbdadfa31f8fb9afaa389358d138cac49bd69e56/centos/9/flavors/default/ 2024-09-13T17:39:49.068 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1627.gbbdadfa3 2024-09-13T17:39:49.165 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/bbdadfa31f8fb9afaa389358d138cac49bd69e56/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/bbdadfa31f8fb9afaa389358d138cac49bd69e56/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/bbdadfa31f8fb9afaa389358d138cac49bd69e56/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-13T17:39:49.165 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-13T17:39:49.165 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-13T17:39:49.188 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/bbdadfa31f8fb9afaa389358d138cac49bd69e56/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/bbdadfa31f8fb9afaa389358d138cac49bd69e56/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/bbdadfa31f8fb9afaa389358d138cac49bd69e56/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-13T17:39:49.189 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T17:39:49.189 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-13T17:39:49.192 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, rbd-fuse, rbd-mirror, rbd-nbd, python-rados, python-rgw, python-rbd, python-cephfs, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux on remote rpm x86_64 2024-09-13T17:39:49.192 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T17:39:49.192 INFO:teuthology.packaging:ref: None 2024-09-13T17:39:49.192 INFO:teuthology.packaging:tag: None 2024-09-13T17:39:49.192 INFO:teuthology.packaging:branch: quincy 2024-09-13T17:39:49.192 INFO:teuthology.packaging:sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:39:49.192 DEBUG:teuthology.orchestra.run.smithi102:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-13T17:39:49.215 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, rbd-fuse, rbd-mirror, rbd-nbd, python-rados, python-rgw, python-rbd, python-cephfs, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux on remote rpm x86_64 2024-09-13T17:39:49.215 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T17:39:49.215 INFO:teuthology.packaging:ref: None 2024-09-13T17:39:49.215 INFO:teuthology.packaging:tag: None 2024-09-13T17:39:49.215 INFO:teuthology.packaging:branch: quincy 2024-09-13T17:39:49.215 INFO:teuthology.packaging:sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:39:49.215 DEBUG:teuthology.orchestra.run.smithi171:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-13T17:39:49.259 DEBUG:teuthology.orchestra.run.smithi102:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-13T17:39:49.282 DEBUG:teuthology.orchestra.run.smithi171:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-13T17:39:49.335 DEBUG:teuthology.orchestra.run.smithi102:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-13T17:39:49.358 DEBUG:teuthology.orchestra.run.smithi171:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-13T17:39:49.399 INFO:teuthology.orchestra.run.smithi102.stdout:check_obsoletes = 1 2024-09-13T17:39:49.401 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum clean all 2024-09-13T17:39:49.422 INFO:teuthology.orchestra.run.smithi171.stdout:check_obsoletes = 1 2024-09-13T17:39:49.424 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum clean all 2024-09-13T17:39:49.633 INFO:teuthology.orchestra.run.smithi102.stdout:45 files removed 2024-09-13T17:39:49.659 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 rbd-fuse rbd-mirror rbd-nbd python-rados python-rgw python-rbd python-cephfs bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux 2024-09-13T17:39:49.660 INFO:teuthology.orchestra.run.smithi171.stdout:45 files removed 2024-09-13T17:39:49.686 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 rbd-fuse rbd-mirror rbd-nbd python-rados python-rgw python-rbd python-cephfs bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux 2024-09-13T17:39:50.206 INFO:teuthology.orchestra.run.smithi102.stdout:ceph packages for x86_64 290 kB/s | 77 kB 00:00 2024-09-13T17:39:50.252 INFO:teuthology.orchestra.run.smithi171.stdout:ceph packages for x86_64 275 kB/s | 77 kB 00:00 2024-09-13T17:39:50.482 INFO:teuthology.orchestra.run.smithi102.stdout:ceph noarch packages 46 kB/s | 11 kB 00:00 2024-09-13T17:39:50.545 INFO:teuthology.orchestra.run.smithi171.stdout:ceph noarch packages 43 kB/s | 11 kB 00:00 2024-09-13T17:39:50.749 INFO:teuthology.orchestra.run.smithi102.stdout:ceph source packages 7.7 kB/s | 1.8 kB 00:00 2024-09-13T17:39:50.821 INFO:teuthology.orchestra.run.smithi171.stdout:ceph source packages 7.5 kB/s | 1.8 kB 00:00 2024-09-13T17:39:51.283 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.2 MB 00:00 2024-09-13T17:39:51.389 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.2 MB 00:00 2024-09-13T17:39:53.356 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - AppStream 23 MB/s | 20 MB 00:00 2024-09-13T17:39:53.695 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - AppStream 16 MB/s | 20 MB 00:01 2024-09-13T17:39:57.600 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-13T17:39:58.152 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - CRB 9.7 MB/s | 6.5 MB 00:00 2024-09-13T17:39:58.992 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - Extras packages 103 kB/s | 19 kB 00:00 2024-09-13T17:39:59.595 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - Extras packages 84 kB/s | 19 kB 00:00 2024-09-13T17:40:00.497 INFO:teuthology.orchestra.run.smithi171.stdout:Extra Packages for Enterprise Linux 16 MB/s | 23 MB 00:01 2024-09-13T17:40:00.857 INFO:teuthology.orchestra.run.smithi102.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-13T17:40:05.878 INFO:teuthology.orchestra.run.smithi171.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-13T17:40:06.513 INFO:teuthology.orchestra.run.smithi102.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-13T17:40:08.244 INFO:teuthology.orchestra.run.smithi171.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-13T17:40:08.245 INFO:teuthology.orchestra.run.smithi171.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-13T17:40:08.265 INFO:teuthology.orchestra.run.smithi171.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-13T17:40:08.266 INFO:teuthology.orchestra.run.smithi171.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-13T17:40:08.267 INFO:teuthology.orchestra.run.smithi171.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-13T17:40:08.321 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-13T17:40:08.329 INFO:teuthology.orchestra.run.smithi171.stdout:======================================================================================= 2024-09-13T17:40:08.329 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-09-13T17:40:08.329 INFO:teuthology.orchestra.run.smithi171.stdout:======================================================================================= 2024-09-13T17:40:08.329 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-13T17:40:08.329 INFO:teuthology.orchestra.run.smithi171.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-13T17:40:08.329 INFO:teuthology.orchestra.run.smithi171.stdout: ceph x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 6.5 k 2024-09-13T17:40:08.329 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-base x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 5.0 M 2024-09-13T17:40:08.330 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-fuse x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 813 k 2024-09-13T17:40:08.330 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 1.4 M 2024-09-13T17:40:08.330 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1627.gbbdadfa3.el9 ceph-noarch 3.4 M 2024-09-13T17:40:08.330 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1627.gbbdadfa3.el9 ceph-noarch 7.4 M 2024-09-13T17:40:08.330 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-rook noarch 2:17.2.7-1627.gbbdadfa3.el9 ceph-noarch 47 k 2024-09-13T17:40:08.330 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-radosgw x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 14 M 2024-09-13T17:40:08.330 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-test x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 48 M 2024-09-13T17:40:08.330 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-13T17:40:08.331 INFO:teuthology.orchestra.run.smithi171.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-13T17:40:08.331 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs-devel x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 25 k 2024-09-13T17:40:08.331 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs2 x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 637 k 2024-09-13T17:40:08.331 INFO:teuthology.orchestra.run.smithi171.stdout: librados-devel x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 126 k 2024-09-13T17:40:08.331 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-13T17:40:08.331 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cephfs x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 156 k 2024-09-13T17:40:08.331 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rados x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 320 k 2024-09-13T17:40:08.332 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rbd x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 294 k 2024-09-13T17:40:08.332 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rgw x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 99 k 2024-09-13T17:40:08.332 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-fuse x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 82 k 2024-09-13T17:40:08.332 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-mirror x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 2.9 M 2024-09-13T17:40:08.332 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-nbd x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 165 k 2024-09-13T17:40:08.332 INFO:teuthology.orchestra.run.smithi171.stdout:Upgrading: 2024-09-13T17:40:08.332 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-13T17:40:08.332 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-13T17:40:08.333 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-13T17:40:08.333 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-13T17:40:08.333 INFO:teuthology.orchestra.run.smithi171.stdout: librados2 x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 3.1 M 2024-09-13T17:40:08.333 INFO:teuthology.orchestra.run.smithi171.stdout: librbd1 x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 2.9 M 2024-09-13T17:40:08.333 INFO:teuthology.orchestra.run.smithi171.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-13T17:40:08.333 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-13T17:40:08.333 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-09-13T17:40:08.333 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-common x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 22 M 2024-09-13T17:40:08.333 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1627.gbbdadfa3.el9 ceph-noarch 24 k 2024-09-13T17:40:08.334 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mds x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 2.1 M 2024-09-13T17:40:08.334 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1627.gbbdadfa3.el9 ceph-noarch 240 k 2024-09-13T17:40:08.334 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mon x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 3.9 M 2024-09-13T17:40:08.334 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-osd x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 15 M 2024-09-13T17:40:08.334 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1627.gbbdadfa3.el9 ceph-noarch 15 k 2024-09-13T17:40:08.334 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-selinux x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 25 k 2024-09-13T17:40:08.334 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-13T17:40:08.334 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-13T17:40:08.334 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-13T17:40:08.334 INFO:teuthology.orchestra.run.smithi171.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-13T17:40:08.334 INFO:teuthology.orchestra.run.smithi171.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-13T17:40:08.334 INFO:teuthology.orchestra.run.smithi171.stdout: libcephsqlite x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 163 k 2024-09-13T17:40:08.335 INFO:teuthology.orchestra.run.smithi171.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-13T17:40:08.335 INFO:teuthology.orchestra.run.smithi171.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-13T17:40:08.335 INFO:teuthology.orchestra.run.smithi171.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-13T17:40:08.335 INFO:teuthology.orchestra.run.smithi171.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-13T17:40:08.335 INFO:teuthology.orchestra.run.smithi171.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-13T17:40:08.335 INFO:teuthology.orchestra.run.smithi171.stdout: libradosstriper1 x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 457 k 2024-09-13T17:40:08.335 INFO:teuthology.orchestra.run.smithi171.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-13T17:40:08.335 INFO:teuthology.orchestra.run.smithi171.stdout: librgw2 x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 4.9 M 2024-09-13T17:40:08.335 INFO:teuthology.orchestra.run.smithi171.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-13T17:40:08.336 INFO:teuthology.orchestra.run.smithi171.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-13T17:40:08.336 INFO:teuthology.orchestra.run.smithi171.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-13T17:40:08.336 INFO:teuthology.orchestra.run.smithi171.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-13T17:40:08.336 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-13T17:40:08.336 INFO:teuthology.orchestra.run.smithi171.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-13T17:40:08.336 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-13T17:40:08.336 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-argparse x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 45 k 2024-09-13T17:40:08.336 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-common x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 120 k 2024-09-13T17:40:08.337 INFO:teuthology.orchestra.run.smithi171.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-13T17:40:08.337 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-13T17:40:08.337 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-13T17:40:08.337 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-13T17:40:08.337 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-13T17:40:08.337 INFO:teuthology.orchestra.run.smithi171.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-13T17:40:08.337 INFO:teuthology.orchestra.run.smithi171.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-13T17:40:08.337 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-13T17:40:08.337 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-13T17:40:08.337 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-13T17:40:08.338 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-13T17:40:08.338 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-13T17:40:08.338 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-13T17:40:08.338 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-13T17:40:08.338 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-13T17:40:08.338 INFO:teuthology.orchestra.run.smithi171.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-13T17:40:08.338 INFO:teuthology.orchestra.run.smithi171.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-13T17:40:08.338 INFO:teuthology.orchestra.run.smithi171.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-13T17:40:08.338 INFO:teuthology.orchestra.run.smithi171.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-13T17:40:08.339 INFO:teuthology.orchestra.run.smithi171.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-13T17:40:08.339 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-13T17:40:08.339 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-13T17:40:08.339 INFO:teuthology.orchestra.run.smithi171.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-13T17:40:08.339 INFO:teuthology.orchestra.run.smithi171.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-13T17:40:08.339 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-13T17:40:08.340 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-13T17:40:08.340 INFO:teuthology.orchestra.run.smithi171.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-13T17:40:08.340 INFO:teuthology.orchestra.run.smithi171.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-13T17:40:08.340 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-13T17:40:08.340 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-13T17:40:08.340 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-13T17:40:08.340 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-13T17:40:08.340 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-13T17:40:08.340 INFO:teuthology.orchestra.run.smithi171.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-13T17:40:08.340 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-13T17:40:08.341 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-13T17:40:08.341 INFO:teuthology.orchestra.run.smithi171.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-13T17:40:08.341 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-13T17:40:08.341 INFO:teuthology.orchestra.run.smithi171.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-13T17:40:08.341 INFO:teuthology.orchestra.run.smithi171.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-13T17:40:08.341 INFO:teuthology.orchestra.run.smithi171.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-13T17:40:08.341 INFO:teuthology.orchestra.run.smithi171.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-13T17:40:08.341 INFO:teuthology.orchestra.run.smithi171.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-13T17:40:08.341 INFO:teuthology.orchestra.run.smithi171.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-13T17:40:08.341 INFO:teuthology.orchestra.run.smithi171.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-13T17:40:08.341 INFO:teuthology.orchestra.run.smithi171.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-13T17:40:08.341 INFO:teuthology.orchestra.run.smithi171.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-13T17:40:08.342 INFO:teuthology.orchestra.run.smithi171.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-13T17:40:08.342 INFO:teuthology.orchestra.run.smithi171.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-13T17:40:08.342 INFO:teuthology.orchestra.run.smithi171.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-13T17:40:08.342 INFO:teuthology.orchestra.run.smithi171.stdout:Installing weak dependencies: 2024-09-13T17:40:08.342 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-13T17:40:08.342 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:40:08.342 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-13T17:40:08.342 INFO:teuthology.orchestra.run.smithi171.stdout:======================================================================================= 2024-09-13T17:40:08.342 INFO:teuthology.orchestra.run.smithi171.stdout:Install 105 Packages 2024-09-13T17:40:08.342 INFO:teuthology.orchestra.run.smithi171.stdout:Upgrade 8 Packages 2024-09-13T17:40:08.342 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:40:08.343 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 196 M 2024-09-13T17:40:08.343 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-13T17:40:08.854 INFO:teuthology.orchestra.run.smithi102.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-13T17:40:08.854 INFO:teuthology.orchestra.run.smithi102.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-13T17:40:08.875 INFO:teuthology.orchestra.run.smithi102.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-13T17:40:08.876 INFO:teuthology.orchestra.run.smithi102.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-13T17:40:08.877 INFO:teuthology.orchestra.run.smithi102.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-13T17:40:08.913 INFO:teuthology.orchestra.run.smithi171.stdout:(1/113): ceph-17.2.7-1627.gbbdadfa3.el9.x86_64. 54 kB/s | 6.5 kB 00:00 2024-09-13T17:40:08.931 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-13T17:40:08.946 INFO:teuthology.orchestra.run.smithi102.stdout:======================================================================================= 2024-09-13T17:40:08.946 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-13T17:40:08.946 INFO:teuthology.orchestra.run.smithi102.stdout:======================================================================================= 2024-09-13T17:40:08.946 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-13T17:40:08.946 INFO:teuthology.orchestra.run.smithi102.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-13T17:40:08.946 INFO:teuthology.orchestra.run.smithi102.stdout: ceph x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 6.5 k 2024-09-13T17:40:08.946 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 5.0 M 2024-09-13T17:40:08.946 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 813 k 2024-09-13T17:40:08.946 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 1.4 M 2024-09-13T17:40:08.946 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1627.gbbdadfa3.el9 ceph-noarch 3.4 M 2024-09-13T17:40:08.947 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1627.gbbdadfa3.el9 ceph-noarch 7.4 M 2024-09-13T17:40:08.947 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-rook noarch 2:17.2.7-1627.gbbdadfa3.el9 ceph-noarch 47 k 2024-09-13T17:40:08.947 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 14 M 2024-09-13T17:40:08.947 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 48 M 2024-09-13T17:40:08.947 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-13T17:40:08.947 INFO:teuthology.orchestra.run.smithi102.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-13T17:40:08.947 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs-devel x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 25 k 2024-09-13T17:40:08.947 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2 x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 637 k 2024-09-13T17:40:08.947 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 126 k 2024-09-13T17:40:08.947 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-13T17:40:08.947 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 156 k 2024-09-13T17:40:08.948 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 320 k 2024-09-13T17:40:08.948 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 294 k 2024-09-13T17:40:08.948 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 99 k 2024-09-13T17:40:08.948 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 82 k 2024-09-13T17:40:08.948 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 2.9 M 2024-09-13T17:40:08.948 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 165 k 2024-09-13T17:40:08.948 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrading: 2024-09-13T17:40:08.949 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-13T17:40:08.949 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-13T17:40:08.949 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-13T17:40:08.949 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-13T17:40:08.949 INFO:teuthology.orchestra.run.smithi102.stdout: librados2 x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 3.1 M 2024-09-13T17:40:08.949 INFO:teuthology.orchestra.run.smithi102.stdout: librbd1 x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 2.9 M 2024-09-13T17:40:08.949 INFO:teuthology.orchestra.run.smithi102.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-13T17:40:08.949 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-13T17:40:08.949 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-13T17:40:08.949 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 22 M 2024-09-13T17:40:08.950 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1627.gbbdadfa3.el9 ceph-noarch 24 k 2024-09-13T17:40:08.950 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 2.1 M 2024-09-13T17:40:08.950 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1627.gbbdadfa3.el9 ceph-noarch 240 k 2024-09-13T17:40:08.950 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 3.9 M 2024-09-13T17:40:08.950 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 15 M 2024-09-13T17:40:08.950 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1627.gbbdadfa3.el9 ceph-noarch 15 k 2024-09-13T17:40:08.950 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-selinux x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 25 k 2024-09-13T17:40:08.950 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-13T17:40:08.950 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-13T17:40:08.950 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-13T17:40:08.950 INFO:teuthology.orchestra.run.smithi102.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-13T17:40:08.950 INFO:teuthology.orchestra.run.smithi102.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-13T17:40:08.950 INFO:teuthology.orchestra.run.smithi102.stdout: libcephsqlite x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 163 k 2024-09-13T17:40:08.951 INFO:teuthology.orchestra.run.smithi102.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-13T17:40:08.951 INFO:teuthology.orchestra.run.smithi102.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-13T17:40:08.951 INFO:teuthology.orchestra.run.smithi102.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-13T17:40:08.951 INFO:teuthology.orchestra.run.smithi102.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-13T17:40:08.951 INFO:teuthology.orchestra.run.smithi102.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-13T17:40:08.951 INFO:teuthology.orchestra.run.smithi102.stdout: libradosstriper1 x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 457 k 2024-09-13T17:40:08.951 INFO:teuthology.orchestra.run.smithi102.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-13T17:40:08.951 INFO:teuthology.orchestra.run.smithi102.stdout: librgw2 x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 4.9 M 2024-09-13T17:40:08.951 INFO:teuthology.orchestra.run.smithi102.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-13T17:40:08.951 INFO:teuthology.orchestra.run.smithi102.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-13T17:40:08.952 INFO:teuthology.orchestra.run.smithi102.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-13T17:40:08.952 INFO:teuthology.orchestra.run.smithi102.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-13T17:40:08.952 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-13T17:40:08.952 INFO:teuthology.orchestra.run.smithi102.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-13T17:40:08.952 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-13T17:40:08.952 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-argparse x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 45 k 2024-09-13T17:40:08.952 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-common x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 120 k 2024-09-13T17:40:08.952 INFO:teuthology.orchestra.run.smithi102.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-13T17:40:08.952 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-13T17:40:08.952 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-13T17:40:08.952 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-13T17:40:08.953 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-13T17:40:08.953 INFO:teuthology.orchestra.run.smithi102.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-13T17:40:08.953 INFO:teuthology.orchestra.run.smithi102.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-13T17:40:08.953 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-13T17:40:08.953 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-13T17:40:08.953 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-13T17:40:08.953 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-13T17:40:08.953 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-13T17:40:08.953 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-13T17:40:08.953 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-13T17:40:08.953 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-13T17:40:08.954 INFO:teuthology.orchestra.run.smithi102.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-13T17:40:08.954 INFO:teuthology.orchestra.run.smithi102.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-13T17:40:08.954 INFO:teuthology.orchestra.run.smithi102.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-13T17:40:08.954 INFO:teuthology.orchestra.run.smithi102.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-13T17:40:08.954 INFO:teuthology.orchestra.run.smithi102.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-13T17:40:08.954 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-13T17:40:08.954 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-13T17:40:08.954 INFO:teuthology.orchestra.run.smithi102.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-13T17:40:08.954 INFO:teuthology.orchestra.run.smithi102.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-13T17:40:08.955 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-13T17:40:08.955 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-13T17:40:08.955 INFO:teuthology.orchestra.run.smithi102.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-13T17:40:08.955 INFO:teuthology.orchestra.run.smithi102.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-13T17:40:08.955 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-13T17:40:08.955 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-13T17:40:08.955 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-13T17:40:08.955 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-13T17:40:08.955 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-13T17:40:08.955 INFO:teuthology.orchestra.run.smithi102.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-13T17:40:08.955 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-13T17:40:08.955 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-13T17:40:08.955 INFO:teuthology.orchestra.run.smithi102.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-13T17:40:08.956 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-13T17:40:08.956 INFO:teuthology.orchestra.run.smithi102.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-13T17:40:08.956 INFO:teuthology.orchestra.run.smithi102.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-13T17:40:08.956 INFO:teuthology.orchestra.run.smithi102.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-13T17:40:08.956 INFO:teuthology.orchestra.run.smithi102.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-13T17:40:08.956 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-13T17:40:08.956 INFO:teuthology.orchestra.run.smithi102.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-13T17:40:08.956 INFO:teuthology.orchestra.run.smithi102.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-13T17:40:08.957 INFO:teuthology.orchestra.run.smithi102.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-13T17:40:08.957 INFO:teuthology.orchestra.run.smithi102.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-13T17:40:08.957 INFO:teuthology.orchestra.run.smithi102.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-13T17:40:08.957 INFO:teuthology.orchestra.run.smithi102.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-13T17:40:08.957 INFO:teuthology.orchestra.run.smithi102.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-13T17:40:08.957 INFO:teuthology.orchestra.run.smithi102.stdout:Installing weak dependencies: 2024-09-13T17:40:08.957 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-13T17:40:08.957 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:40:08.957 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-13T17:40:08.957 INFO:teuthology.orchestra.run.smithi102.stdout:======================================================================================= 2024-09-13T17:40:08.957 INFO:teuthology.orchestra.run.smithi102.stdout:Install 105 Packages 2024-09-13T17:40:08.957 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrade 8 Packages 2024-09-13T17:40:08.958 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:40:08.958 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 196 M 2024-09-13T17:40:08.958 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-13T17:40:09.165 INFO:teuthology.orchestra.run.smithi171.stdout:(2/113): ceph-fuse-17.2.7-1627.gbbdadfa3.el9.x8 3.2 MB/s | 813 kB 00:00 2024-09-13T17:40:09.239 INFO:teuthology.orchestra.run.smithi171.stdout:(3/113): ceph-base-17.2.7-1627.gbbdadfa3.el9.x8 11 MB/s | 5.0 MB 00:00 2024-09-13T17:40:09.290 INFO:teuthology.orchestra.run.smithi102.stdout:(1/113): ceph-17.2.7-1627.gbbdadfa3.el9.x86_64. 41 kB/s | 6.5 kB 00:00 2024-09-13T17:40:09.323 INFO:teuthology.orchestra.run.smithi171.stdout:(4/113): ceph-mgr-17.2.7-1627.gbbdadfa3.el9.x86 17 MB/s | 1.4 MB 00:00 2024-09-13T17:40:09.556 INFO:teuthology.orchestra.run.smithi171.stdout:(5/113): ceph-mds-17.2.7-1627.gbbdadfa3.el9.x86 5.3 MB/s | 2.1 MB 00:00 2024-09-13T17:40:09.623 INFO:teuthology.orchestra.run.smithi171.stdout:(6/113): ceph-mon-17.2.7-1627.gbbdadfa3.el9.x86 13 MB/s | 3.9 MB 00:00 2024-09-13T17:40:09.848 INFO:teuthology.orchestra.run.smithi102.stdout:(2/113): ceph-fuse-17.2.7-1627.gbbdadfa3.el9.x8 1.4 MB/s | 813 kB 00:00 2024-09-13T17:40:10.048 INFO:teuthology.orchestra.run.smithi171.stdout:(7/113): ceph-common-17.2.7-1627.gbbdadfa3.el9. 18 MB/s | 22 MB 00:01 2024-09-13T17:40:10.091 INFO:teuthology.orchestra.run.smithi171.stdout:(8/113): ceph-selinux-17.2.7-1627.gbbdadfa3.el9 605 kB/s | 25 kB 00:00 2024-09-13T17:40:10.382 INFO:teuthology.orchestra.run.smithi102.stdout:(3/113): ceph-mds-17.2.7-1627.gbbdadfa3.el9.x86 3.9 MB/s | 2.1 MB 00:00 2024-09-13T17:40:10.433 INFO:teuthology.orchestra.run.smithi171.stdout:(9/113): ceph-radosgw-17.2.7-1627.gbbdadfa3.el9 18 MB/s | 14 MB 00:00 2024-09-13T17:40:10.483 INFO:teuthology.orchestra.run.smithi171.stdout:(10/113): libcephfs-devel-17.2.7-1627.gbbdadfa3 500 kB/s | 25 kB 00:00 2024-09-13T17:40:10.565 INFO:teuthology.orchestra.run.smithi102.stdout:(4/113): ceph-base-17.2.7-1627.gbbdadfa3.el9.x8 3.5 MB/s | 5.0 MB 00:01 2024-09-13T17:40:10.591 INFO:teuthology.orchestra.run.smithi171.stdout:(11/113): libcephfs2-17.2.7-1627.gbbdadfa3.el9. 5.8 MB/s | 637 kB 00:00 2024-09-13T17:40:10.684 INFO:teuthology.orchestra.run.smithi171.stdout:(12/113): libcephsqlite-17.2.7-1627.gbbdadfa3.e 1.7 MB/s | 163 kB 00:00 2024-09-13T17:40:10.715 INFO:teuthology.orchestra.run.smithi102.stdout:(5/113): ceph-mgr-17.2.7-1627.gbbdadfa3.el9.x86 4.2 MB/s | 1.4 MB 00:00 2024-09-13T17:40:10.942 INFO:teuthology.orchestra.run.smithi171.stdout:(13/113): ceph-osd-17.2.7-1627.gbbdadfa3.el9.x8 11 MB/s | 15 MB 00:01 2024-09-13T17:40:10.968 INFO:teuthology.orchestra.run.smithi171.stdout:(14/113): librados-devel-17.2.7-1627.gbbdadfa3. 444 kB/s | 126 kB 00:00 2024-09-13T17:40:11.001 INFO:teuthology.orchestra.run.smithi171.stdout:(15/113): libradosstriper1-17.2.7-1627.gbbdadfa 7.6 MB/s | 457 kB 00:00 2024-09-13T17:40:11.007 INFO:teuthology.orchestra.run.smithi102.stdout:(6/113): ceph-mon-17.2.7-1627.gbbdadfa3.el9.x86 8.9 MB/s | 3.9 MB 00:00 2024-09-13T17:40:11.052 INFO:teuthology.orchestra.run.smithi171.stdout:(16/113): python3-ceph-argparse-17.2.7-1627.gbb 899 kB/s | 45 kB 00:00 2024-09-13T17:40:11.277 INFO:teuthology.orchestra.run.smithi171.stdout:(17/113): librgw2-17.2.7-1627.gbbdadfa3.el9.x86 16 MB/s | 4.9 MB 00:00 2024-09-13T17:40:11.336 INFO:teuthology.orchestra.run.smithi171.stdout:(18/113): python3-cephfs-17.2.7-1627.gbbdadfa3. 2.6 MB/s | 156 kB 00:00 2024-09-13T17:40:11.411 INFO:teuthology.orchestra.run.smithi171.stdout:(19/113): python3-rados-17.2.7-1627.gbbdadfa3.e 4.2 MB/s | 320 kB 00:00 2024-09-13T17:40:11.487 INFO:teuthology.orchestra.run.smithi171.stdout:(20/113): python3-rbd-17.2.7-1627.gbbdadfa3.el9 3.8 MB/s | 294 kB 00:00 2024-09-13T17:40:11.562 INFO:teuthology.orchestra.run.smithi171.stdout:(21/113): python3-rgw-17.2.7-1627.gbbdadfa3.el9 1.3 MB/s | 99 kB 00:00 2024-09-13T17:40:11.588 INFO:teuthology.orchestra.run.smithi171.stdout:(22/113): python3-ceph-common-17.2.7-1627.gbbda 224 kB/s | 120 kB 00:00 2024-09-13T17:40:11.638 INFO:teuthology.orchestra.run.smithi171.stdout:(23/113): rbd-fuse-17.2.7-1627.gbbdadfa3.el9.x8 1.1 MB/s | 82 kB 00:00 2024-09-13T17:40:11.730 INFO:teuthology.orchestra.run.smithi171.stdout:(24/113): rbd-nbd-17.2.7-1627.gbbdadfa3.el9.x86 1.8 MB/s | 165 kB 00:00 2024-09-13T17:40:11.822 INFO:teuthology.orchestra.run.smithi171.stdout:(25/113): ceph-grafana-dashboards-17.2.7-1627.g 266 kB/s | 24 kB 00:00 2024-09-13T17:40:11.989 INFO:teuthology.orchestra.run.smithi171.stdout:(26/113): rbd-mirror-17.2.7-1627.gbbdadfa3.el9. 7.3 MB/s | 2.9 MB 00:00 2024-09-13T17:40:12.148 INFO:teuthology.orchestra.run.smithi171.stdout:(27/113): ceph-mgr-dashboard-17.2.7-1627.gbbdad 10 MB/s | 3.4 MB 00:00 2024-09-13T17:40:12.215 INFO:teuthology.orchestra.run.smithi171.stdout:(28/113): ceph-mgr-modules-core-17.2.7-1627.gbb 3.5 MB/s | 240 kB 00:00 2024-09-13T17:40:12.265 INFO:teuthology.orchestra.run.smithi171.stdout:(29/113): ceph-mgr-rook-17.2.7-1627.gbbdadfa3.e 950 kB/s | 47 kB 00:00 2024-09-13T17:40:12.266 INFO:teuthology.orchestra.run.smithi102.stdout:(7/113): ceph-common-17.2.7-1627.gbbdadfa3.el9. 7.2 MB/s | 22 MB 00:03 2024-09-13T17:40:12.324 INFO:teuthology.orchestra.run.smithi171.stdout:(30/113): ceph-prometheus-alerts-17.2.7-1627.gb 250 kB/s | 15 kB 00:00 2024-09-13T17:40:12.333 INFO:teuthology.orchestra.run.smithi102.stdout:(8/113): ceph-selinux-17.2.7-1627.gbbdadfa3.el9 379 kB/s | 25 kB 00:00 2024-09-13T17:40:12.450 INFO:teuthology.orchestra.run.smithi171.stdout:(31/113): fuse-2.9.9-16.el9.x86_64.rpm 638 kB/s | 80 kB 00:00 2024-09-13T17:40:12.594 INFO:teuthology.orchestra.run.smithi171.stdout:(32/113): ceph-mgr-diskprediction-local-17.2.7- 12 MB/s | 7.4 MB 00:00 2024-09-13T17:40:12.834 INFO:teuthology.orchestra.run.smithi102.stdout:(9/113): ceph-radosgw-17.2.7-1627.gbbdadfa3.el9 7.9 MB/s | 14 MB 00:01 2024-09-13T17:40:12.884 INFO:teuthology.orchestra.run.smithi102.stdout:(10/113): libcephfs-devel-17.2.7-1627.gbbdadfa3 503 kB/s | 25 kB 00:00 2024-09-13T17:40:13.002 INFO:teuthology.orchestra.run.smithi171.stdout:(33/113): ceph-test-17.2.7-1627.gbbdadfa3.el9.x 16 MB/s | 48 MB 00:02 2024-09-13T17:40:13.035 INFO:teuthology.orchestra.run.smithi171.stdout:(34/113): libgfortran-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 801 kB 00:00 2024-09-13T17:40:13.077 INFO:teuthology.orchestra.run.smithi102.stdout:(11/113): ceph-osd-17.2.7-1627.gbbdadfa3.el9.x8 6.5 MB/s | 15 MB 00:02 2024-09-13T17:40:13.101 INFO:teuthology.orchestra.run.smithi102.stdout:(12/113): libcephfs2-17.2.7-1627.gbbdadfa3.el9. 2.9 MB/s | 637 kB 00:00 2024-09-13T17:40:13.134 INFO:teuthology.orchestra.run.smithi102.stdout:(13/113): libcephsqlite-17.2.7-1627.gbbdadfa3.e 2.7 MB/s | 163 kB 00:00 2024-09-13T17:40:13.152 INFO:teuthology.orchestra.run.smithi171.stdout:(35/113): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-09-13T17:40:13.186 INFO:teuthology.orchestra.run.smithi171.stdout:(36/113): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-09-13T17:40:13.191 INFO:teuthology.orchestra.run.smithi102.stdout:(14/113): librados-devel-17.2.7-1627.gbbdadfa3. 1.4 MB/s | 126 kB 00:00 2024-09-13T17:40:13.227 INFO:teuthology.orchestra.run.smithi171.stdout:(37/113): libquadmath-11.5.0-2.el9.x86_64.rpm 301 kB/s | 191 kB 00:00 2024-09-13T17:40:13.243 INFO:teuthology.orchestra.run.smithi102.stdout:(15/113): libradosstriper1-17.2.7-1627.gbbdadfa 4.1 MB/s | 457 kB 00:00 2024-09-13T17:40:13.252 INFO:teuthology.orchestra.run.smithi171.stdout:(38/113): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-09-13T17:40:13.278 INFO:teuthology.orchestra.run.smithi171.stdout:(39/113): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-09-13T17:40:13.302 INFO:teuthology.orchestra.run.smithi102.stdout:(16/113): python3-ceph-argparse-17.2.7-1627.gbb 776 kB/s | 45 kB 00:00 2024-09-13T17:40:13.320 INFO:teuthology.orchestra.run.smithi171.stdout:(40/113): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-09-13T17:40:13.345 INFO:teuthology.orchestra.run.smithi171.stdout:(41/113): python3-urllib3-1.26.5-6.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-09-13T17:40:13.369 INFO:teuthology.orchestra.run.smithi102.stdout:(17/113): python3-ceph-common-17.2.7-1627.gbbda 1.8 MB/s | 120 kB 00:00 2024-09-13T17:40:13.395 INFO:teuthology.orchestra.run.smithi171.stdout:(42/113): elfutils-libelf-devel-0.191-4.el9.x86 288 kB/s | 22 kB 00:00 2024-09-13T17:40:13.427 INFO:teuthology.orchestra.run.smithi102.stdout:(18/113): python3-cephfs-17.2.7-1627.gbbdadfa3. 2.6 MB/s | 156 kB 00:00 2024-09-13T17:40:13.446 INFO:teuthology.orchestra.run.smithi171.stdout:(43/113): bison-3.7.4-5.el9.x86_64.rpm 5.5 MB/s | 943 kB 00:00 2024-09-13T17:40:13.471 INFO:teuthology.orchestra.run.smithi171.stdout:(44/113): flexiblas-3.0.4-8.el9.x86_64.rpm 434 kB/s | 33 kB 00:00 2024-09-13T17:40:13.511 INFO:teuthology.orchestra.run.smithi102.stdout:(19/113): python3-rados-17.2.7-1627.gbbdadfa3.e 3.7 MB/s | 320 kB 00:00 2024-09-13T17:40:13.521 INFO:teuthology.orchestra.run.smithi171.stdout:(45/113): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-09-13T17:40:13.547 INFO:teuthology.orchestra.run.smithi171.stdout:(46/113): flex-2.6.4-9.el9.x86_64.rpm 1.5 MB/s | 313 kB 00:00 2024-09-13T17:40:13.578 INFO:teuthology.orchestra.run.smithi102.stdout:(20/113): librgw2-17.2.7-1627.gbbdadfa3.el9.x86 13 MB/s | 4.9 MB 00:00 2024-09-13T17:40:13.597 INFO:teuthology.orchestra.run.smithi171.stdout:(47/113): flexiblas-netlib-3.0.4-8.el9.x86_64.r 20 MB/s | 3.0 MB 00:00 2024-09-13T17:40:13.623 INFO:teuthology.orchestra.run.smithi171.stdout:(48/113): librabbitmq-0.11.0-7.el9.x86_64.rpm 599 kB/s | 45 kB 00:00 2024-09-13T17:40:13.629 INFO:teuthology.orchestra.run.smithi102.stdout:(21/113): python3-rbd-17.2.7-1627.gbbdadfa3.el9 2.5 MB/s | 294 kB 00:00 2024-09-13T17:40:13.648 INFO:teuthology.orchestra.run.smithi171.stdout:(49/113): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-09-13T17:40:13.654 INFO:teuthology.orchestra.run.smithi102.stdout:(22/113): python3-rgw-17.2.7-1627.gbbdadfa3.el9 1.3 MB/s | 99 kB 00:00 2024-09-13T17:40:13.674 INFO:teuthology.orchestra.run.smithi171.stdout:(50/113): libzstd-devel-1.5.1-2.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-09-13T17:40:13.687 INFO:teuthology.orchestra.run.smithi102.stdout:(23/113): rbd-fuse-17.2.7-1627.gbbdadfa3.el9.x8 1.4 MB/s | 82 kB 00:00 2024-09-13T17:40:13.715 INFO:teuthology.orchestra.run.smithi171.stdout:(51/113): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-09-13T17:40:13.741 INFO:teuthology.orchestra.run.smithi171.stdout:(52/113): openblas-0.3.26-2.el9.x86_64.rpm 576 kB/s | 38 kB 00:00 2024-09-13T17:40:13.746 INFO:teuthology.orchestra.run.smithi102.stdout:(24/113): rbd-nbd-17.2.7-1627.gbbdadfa3.el9.x86 2.8 MB/s | 165 kB 00:00 2024-09-13T17:40:13.766 INFO:teuthology.orchestra.run.smithi171.stdout:(53/113): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-13T17:40:13.805 INFO:teuthology.orchestra.run.smithi102.stdout:(25/113): ceph-grafana-dashboards-17.2.7-1627.g 418 kB/s | 24 kB 00:00 2024-09-13T17:40:13.816 INFO:teuthology.orchestra.run.smithi171.stdout:(54/113): python3-devel-3.9.19-8.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-09-13T17:40:13.884 INFO:teuthology.orchestra.run.smithi171.stdout:(55/113): openblas-openmp-0.3.26-2.el9.x86_64.r 29 MB/s | 4.9 MB 00:00 2024-09-13T17:40:13.901 INFO:teuthology.orchestra.run.smithi171.stdout:(56/113): python3-jsonpatch-1.21-16.el9.noarch. 305 kB/s | 26 kB 00:00 2024-09-13T17:40:13.921 INFO:teuthology.orchestra.run.smithi102.stdout:(26/113): rbd-mirror-17.2.7-1627.gbbdadfa3.el9. 11 MB/s | 2.9 MB 00:00 2024-09-13T17:40:13.951 INFO:teuthology.orchestra.run.smithi171.stdout:(57/113): python3-jsonpointer-2.0-4.el9.noarch. 286 kB/s | 19 kB 00:00 2024-09-13T17:40:13.977 INFO:teuthology.orchestra.run.smithi171.stdout:(58/113): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-09-13T17:40:14.002 INFO:teuthology.orchestra.run.smithi171.stdout:(59/113): python3-markupsafe-1.1.1-12.el9.x86_6 689 kB/s | 35 kB 00:00 2024-09-13T17:40:14.044 INFO:teuthology.orchestra.run.smithi171.stdout:(60/113): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-09-13T17:40:14.094 INFO:teuthology.orchestra.run.smithi171.stdout:(61/113): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-09-13T17:40:14.137 INFO:teuthology.orchestra.run.smithi171.stdout:(62/113): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-09-13T17:40:14.178 INFO:teuthology.orchestra.run.smithi171.stdout:(63/113): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-13T17:40:14.221 INFO:teuthology.orchestra.run.smithi171.stdout:(64/113): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.7 MB/s | 159 kB 00:00 2024-09-13T17:40:14.239 INFO:teuthology.orchestra.run.smithi102.stdout:(27/113): ceph-mgr-dashboard-17.2.7-1627.gbbdad 7.7 MB/s | 3.4 MB 00:00 2024-09-13T17:40:14.289 INFO:teuthology.orchestra.run.smithi102.stdout:(28/113): ceph-mgr-modules-core-17.2.7-1627.gbb 4.7 MB/s | 240 kB 00:00 2024-09-13T17:40:14.298 INFO:teuthology.orchestra.run.smithi171.stdout:(65/113): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-09-13T17:40:14.321 INFO:teuthology.orchestra.run.smithi171.stdout:(66/113): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-09-13T17:40:14.347 INFO:teuthology.orchestra.run.smithi171.stdout:(67/113): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-13T17:40:14.356 INFO:teuthology.orchestra.run.smithi102.stdout:(29/113): ceph-mgr-rook-17.2.7-1627.gbbdadfa3.e 714 kB/s | 47 kB 00:00 2024-09-13T17:40:14.372 INFO:teuthology.orchestra.run.smithi171.stdout:(68/113): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-13T17:40:14.439 INFO:teuthology.orchestra.run.smithi171.stdout:(69/113): openssl-devel-3.2.2-6.el9.x86_64.rpm 6.3 MB/s | 4.4 MB 00:00 2024-09-13T17:40:14.464 INFO:teuthology.orchestra.run.smithi171.stdout:(70/113): python3-toml-0.10.2-6.el9.noarch.rpm 453 kB/s | 42 kB 00:00 2024-09-13T17:40:14.490 INFO:teuthology.orchestra.run.smithi171.stdout:(71/113): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-13T17:40:14.515 INFO:teuthology.orchestra.run.smithi171.stdout:(72/113): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-13T17:40:14.541 INFO:teuthology.orchestra.run.smithi171.stdout:(73/113): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-09-13T17:40:14.599 INFO:teuthology.orchestra.run.smithi171.stdout:(74/113): liboath-2.6.7-2.el9.x86_64.rpm 834 kB/s | 49 kB 00:00 2024-09-13T17:40:14.633 INFO:teuthology.orchestra.run.smithi171.stdout:(75/113): gperftools-libs-2.9.1-3.el9.x86_64.rp 2.6 MB/s | 308 kB 00:00 2024-09-13T17:40:14.658 INFO:teuthology.orchestra.run.smithi171.stdout:(76/113): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-09-13T17:40:14.692 INFO:teuthology.orchestra.run.smithi171.stdout:(77/113): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.3 MB/s | 43 kB 00:00 2024-09-13T17:40:14.717 INFO:teuthology.orchestra.run.smithi171.stdout:(78/113): python3-cachetools-4.2.4-1.el9.noarch 547 kB/s | 32 kB 00:00 2024-09-13T17:40:14.743 INFO:teuthology.orchestra.run.smithi171.stdout:(79/113): python3-certifi-2023.05.07-4.el9.noar 280 kB/s | 14 kB 00:00 2024-09-13T17:40:14.769 INFO:teuthology.orchestra.run.smithi171.stdout:(80/113): python3-cheroot-8.6.0-4.el9.noarch.rp 3.3 MB/s | 172 kB 00:00 2024-09-13T17:40:14.793 INFO:teuthology.orchestra.run.smithi171.stdout:(81/113): python3-cherrypy-18.6.1-2.el9.noarch. 6.9 MB/s | 358 kB 00:00 2024-09-13T17:40:14.799 INFO:teuthology.orchestra.run.smithi102.stdout:(30/113): ceph-test-17.2.7-1627.gbbdadfa3.el9.x 19 MB/s | 48 MB 00:02 2024-09-13T17:40:14.819 INFO:teuthology.orchestra.run.smithi171.stdout:(82/113): python3-jaraco-8.2.1-3.el9.noarch.rpm 426 kB/s | 11 kB 00:00 2024-09-13T17:40:14.823 INFO:teuthology.orchestra.run.smithi102.stdout:(31/113): ceph-prometheus-alerts-17.2.7-1627.gb 31 kB/s | 15 kB 00:00 2024-09-13T17:40:14.836 INFO:teuthology.orchestra.run.smithi171.stdout:(83/113): python3-jaraco-classes-3.2.1-5.el9.no 1.0 MB/s | 18 kB 00:00 2024-09-13T17:40:14.861 INFO:teuthology.orchestra.run.smithi171.stdout:(84/113): python3-google-auth-2.34.0-1.el9.noar 2.3 MB/s | 220 kB 00:00 2024-09-13T17:40:14.878 INFO:teuthology.orchestra.run.smithi171.stdout:(85/113): python3-jaraco-collections-3.0.0-8.el 550 kB/s | 23 kB 00:00 2024-09-13T17:40:14.895 INFO:teuthology.orchestra.run.smithi171.stdout:(86/113): python3-jaraco-functools-3.5.0-2.el9. 576 kB/s | 19 kB 00:00 2024-09-13T17:40:14.916 INFO:teuthology.orchestra.run.smithi102.stdout:(32/113): ceph-mgr-diskprediction-local-17.2.7- 7.5 MB/s | 7.4 MB 00:00 2024-09-13T17:40:14.921 INFO:teuthology.orchestra.run.smithi171.stdout:(87/113): python3-jaraco-text-3.2.0-6.el9.noarc 466 kB/s | 20 kB 00:00 2024-09-13T17:40:14.954 INFO:teuthology.orchestra.run.smithi171.stdout:(88/113): python3-jwt+crypto-2.4.0-1.el9.noarch 154 kB/s | 9.0 kB 00:00 2024-09-13T17:40:14.980 INFO:teuthology.orchestra.run.smithi171.stdout:(89/113): python3-jwt-2.4.0-1.el9.noarch.rpm 698 kB/s | 41 kB 00:00 2024-09-13T17:40:15.014 INFO:teuthology.orchestra.run.smithi171.stdout:(90/113): python3-logutils-0.3.5-21.el9.noarch. 1.4 MB/s | 46 kB 00:00 2024-09-13T17:40:15.015 INFO:teuthology.orchestra.run.smithi102.stdout:(33/113): fuse-2.9.9-16.el9.x86_64.rpm 368 kB/s | 80 kB 00:00 2024-09-13T17:40:15.041 INFO:teuthology.orchestra.run.smithi102.stdout:(34/113): libquadmath-11.5.0-2.el9.x86_64.rpm 1.5 MB/s | 191 kB 00:00 2024-09-13T17:40:15.047 INFO:teuthology.orchestra.run.smithi171.stdout:(91/113): python3-more-itertools-8.12.0-2.el9.n 2.3 MB/s | 79 kB 00:00 2024-09-13T17:40:15.089 INFO:teuthology.orchestra.run.smithi171.stdout:(92/113): python3-pecan-1.4.2-3.el9.noarch.rpm 6.4 MB/s | 272 kB 00:00 2024-09-13T17:40:15.091 INFO:teuthology.orchestra.run.smithi102.stdout:(35/113): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-09-13T17:40:15.117 INFO:teuthology.orchestra.run.smithi171.stdout:(93/113): python3-portend-3.1.0-2.el9.noarch.rp 659 kB/s | 16 kB 00:00 2024-09-13T17:40:15.118 INFO:teuthology.orchestra.run.smithi102.stdout:(36/113): libgfortran-11.5.0-2.el9.x86_64.rpm 2.7 MB/s | 801 kB 00:00 2024-09-13T17:40:15.142 INFO:teuthology.orchestra.run.smithi102.stdout:(37/113): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-13T17:40:15.157 INFO:teuthology.orchestra.run.smithi171.stdout:(94/113): python3-kubernetes-26.1.0-2.el9.noarc 5.1 MB/s | 1.0 MB 00:00 2024-09-13T17:40:15.167 INFO:teuthology.orchestra.run.smithi102.stdout:(38/113): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-13T17:40:15.182 INFO:teuthology.orchestra.run.smithi171.stdout:(95/113): python3-pyOpenSSL-21.0.0-1.el9.noarch 1.3 MB/s | 90 kB 00:00 2024-09-13T17:40:15.192 INFO:teuthology.orchestra.run.smithi102.stdout:(39/113): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-13T17:40:15.218 INFO:teuthology.orchestra.run.smithi102.stdout:(40/113): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-13T17:40:15.251 INFO:teuthology.orchestra.run.smithi102.stdout:(41/113): python3-cryptography-36.0.1-4.el9.x86 5.8 MB/s | 1.2 MB 00:00 2024-09-13T17:40:15.350 INFO:teuthology.orchestra.run.smithi171.stdout:(96/113): python3-scipy-1.6.2-8.el9.x86_64.rpm 16 MB/s | 16 MB 00:01 2024-09-13T17:40:15.375 INFO:teuthology.orchestra.run.smithi171.stdout:(97/113): python3-repoze-lru-0.7-16.el9.noarch. 141 kB/s | 31 kB 00:00 2024-09-13T17:40:15.409 INFO:teuthology.orchestra.run.smithi171.stdout:(98/113): python3-tempora-5.0.0-2.el9.noarch.rp 1.4 MB/s | 36 kB 00:00 2024-09-13T17:40:15.442 INFO:teuthology.orchestra.run.smithi171.stdout:(99/113): python3-routes-2.5.1-5.el9.noarch.rpm 726 kB/s | 188 kB 00:00 2024-09-13T17:40:15.468 INFO:teuthology.orchestra.run.smithi171.stdout:(100/113): python3-rsa-4.9-2.el9.noarch.rpm 501 kB/s | 59 kB 00:00 2024-09-13T17:40:15.493 INFO:teuthology.orchestra.run.smithi171.stdout:(101/113): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-13T17:40:15.527 INFO:teuthology.orchestra.run.smithi171.stdout:(102/113): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-13T17:40:15.553 INFO:teuthology.orchestra.run.smithi171.stdout:(103/113): python3-zc-lockfile-2.0-10.el9.noarc 341 kB/s | 20 kB 00:00 2024-09-13T17:40:15.595 INFO:teuthology.orchestra.run.smithi171.stdout:(104/113): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-13T17:40:15.670 INFO:teuthology.orchestra.run.smithi171.stdout:(105/113): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-13T17:40:15.751 INFO:teuthology.orchestra.run.smithi102.stdout:(42/113): elfutils-libelf-devel-0.191-4.el9.x86 41 kB/s | 22 kB 00:00 2024-09-13T17:40:15.787 INFO:teuthology.orchestra.run.smithi171.stdout:(106/113): NetworkManager-1.49.90-1.el9.x86_64. 20 MB/s | 2.3 MB 00:00 2024-09-13T17:40:15.887 INFO:teuthology.orchestra.run.smithi171.stdout:(107/113): librados2-17.2.7-1627.gbbdadfa3.el9. 9.2 MB/s | 3.1 MB 00:00 2024-09-13T17:40:15.946 INFO:teuthology.orchestra.run.smithi171.stdout:(108/113): librbd1-17.2.7-1627.gbbdadfa3.el9.x8 8.4 MB/s | 2.9 MB 00:00 2024-09-13T17:40:16.038 INFO:teuthology.orchestra.run.smithi171.stdout:(109/113): NetworkManager-libnm-1.49.90-1.el9.x 7.2 MB/s | 1.8 MB 00:00 2024-09-13T17:40:16.055 INFO:teuthology.orchestra.run.smithi171.stdout:(110/113): NetworkManager-team-1.49.90-1.el9.x8 243 kB/s | 41 kB 00:00 2024-09-13T17:40:16.089 INFO:teuthology.orchestra.run.smithi171.stdout:(111/113): NetworkManager-tui-1.49.90-1.el9.x86 1.7 MB/s | 249 kB 00:00 2024-09-13T17:40:16.102 INFO:teuthology.orchestra.run.smithi102.stdout:(43/113): flex-2.6.4-9.el9.x86_64.rpm 369 kB/s | 313 kB 00:00 2024-09-13T17:40:16.140 INFO:teuthology.orchestra.run.smithi171.stdout:(112/113): openssl-3.2.2-6.el9.x86_64.rpm 13 MB/s | 1.4 MB 00:00 2024-09-13T17:40:16.169 INFO:teuthology.orchestra.run.smithi102.stdout:(44/113): bison-3.7.4-5.el9.x86_64.rpm 966 kB/s | 943 kB 00:00 2024-09-13T17:40:16.311 INFO:teuthology.orchestra.run.smithi102.stdout:(45/113): flexiblas-3.0.4-8.el9.x86_64.rpm 58 kB/s | 33 kB 00:00 2024-09-13T17:40:16.331 INFO:teuthology.orchestra.run.smithi171.stdout:(113/113): openssl-libs-3.2.2-6.el9.x86_64.rpm 8.7 MB/s | 2.4 MB 00:00 2024-09-13T17:40:16.337 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-13T17:40:16.337 INFO:teuthology.orchestra.run.smithi171.stdout:Total 24 MB/s | 196 MB 00:08 2024-09-13T17:40:16.669 INFO:teuthology.orchestra.run.smithi102.stdout:(46/113): flexiblas-openblas-openmp-3.0.4-8.el9 36 kB/s | 18 kB 00:00 2024-09-13T17:40:17.011 INFO:teuthology.orchestra.run.smithi102.stdout:(47/113): libpmemobj-1.12.1-1.el9.x86_64.rpm 229 kB/s | 160 kB 00:00 2024-09-13T17:40:17.166 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-13T17:40:17.253 INFO:teuthology.orchestra.run.smithi102.stdout:(48/113): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.6 MB/s | 3.0 MB 00:01 2024-09-13T17:40:17.286 INFO:teuthology.orchestra.run.smithi102.stdout:(49/113): librabbitmq-0.11.0-7.el9.x86_64.rpm 73 kB/s | 45 kB 00:00 2024-09-13T17:40:17.290 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-13T17:40:17.290 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-13T17:40:17.887 INFO:teuthology.orchestra.run.smithi102.stdout:(50/113): libzstd-devel-1.5.1-2.el9.x86_64.rpm 76 kB/s | 47 kB 00:00 2024-09-13T17:40:17.912 INFO:teuthology.orchestra.run.smithi102.stdout:(51/113): librdkafka-1.6.1-102.el9.x86_64.rpm 736 kB/s | 662 kB 00:00 2024-09-13T17:40:18.121 INFO:teuthology.orchestra.run.smithi102.stdout:(52/113): lttng-ust-2.12.0-6.el9.x86_64.rpm 351 kB/s | 292 kB 00:00 2024-09-13T17:40:18.488 INFO:teuthology.orchestra.run.smithi102.stdout:(53/113): openblas-0.3.26-2.el9.x86_64.rpm 64 kB/s | 38 kB 00:00 2024-09-13T17:40:19.005 INFO:teuthology.orchestra.run.smithi102.stdout:(54/113): openblas-openmp-0.3.26-2.el9.x86_64.r 4.5 MB/s | 4.9 MB 00:01 2024-09-13T17:40:19.255 INFO:teuthology.orchestra.run.smithi102.stdout:(55/113): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.9 MB/s | 4.4 MB 00:01 2024-09-13T17:40:19.267 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-13T17:40:19.267 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-13T17:40:19.289 INFO:teuthology.orchestra.run.smithi102.stdout:(56/113): python3-devel-3.9.19-8.el9.x86_64.rpm 306 kB/s | 245 kB 00:00 2024-09-13T17:40:19.589 INFO:teuthology.orchestra.run.smithi102.stdout:(57/113): python3-jsonpatch-1.21-16.el9.noarch. 44 kB/s | 26 kB 00:00 2024-09-13T17:40:19.756 INFO:teuthology.orchestra.run.smithi102.stdout:(58/113): python3-jsonpointer-2.0-4.el9.noarch. 39 kB/s | 19 kB 00:00 2024-09-13T17:40:20.031 INFO:teuthology.orchestra.run.smithi102.stdout:(59/113): python3-mako-1.1.4-6.el9.noarch.rpm 232 kB/s | 172 kB 00:00 2024-09-13T17:40:20.181 INFO:teuthology.orchestra.run.smithi102.stdout:(60/113): python3-markupsafe-1.1.1-12.el9.x86_6 59 kB/s | 35 kB 00:00 2024-09-13T17:40:20.807 INFO:teuthology.orchestra.run.smithi102.stdout:(61/113): python3-numpy-f2py-1.20.1-5.el9.x86_6 323 kB/s | 250 kB 00:00 2024-09-13T17:40:20.882 INFO:teuthology.orchestra.run.smithi102.stdout:(62/113): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.5 MB/s | 5.1 MB 00:01 2024-09-13T17:40:20.932 INFO:teuthology.orchestra.run.smithi102.stdout:(63/113): python3-oauthlib-3.1.1-5.el9.noarch.r 296 kB/s | 222 kB 00:00 2024-09-13T17:40:21.441 INFO:teuthology.orchestra.run.smithi102.stdout:(64/113): python3-prettytable-0.7.2-27.el9.noar 76 kB/s | 42 kB 00:00 2024-09-13T17:40:21.483 INFO:teuthology.orchestra.run.smithi102.stdout:(65/113): python3-packaging-20.9-5.el9.noarch.r 114 kB/s | 77 kB 00:00 2024-09-13T17:40:21.633 INFO:teuthology.orchestra.run.smithi102.stdout:(66/113): python3-pyasn1-0.4.8-6.el9.noarch.rpm 227 kB/s | 159 kB 00:00 2024-09-13T17:40:22.083 INFO:teuthology.orchestra.run.smithi102.stdout:(67/113): python3-pytz-2021.1-5.el9.noarch.rpm 85 kB/s | 51 kB 00:00 2024-09-13T17:40:22.200 INFO:teuthology.orchestra.run.smithi102.stdout:(68/113): python3-pyasn1-modules-0.4.8-6.el9.no 367 kB/s | 279 kB 00:00 2024-09-13T17:40:22.236 INFO:teuthology.orchestra.run.smithi102.stdout:(69/113): python3-requests-oauthlib-1.3.0-12.el 89 kB/s | 54 kB 00:00 2024-09-13T17:40:22.759 INFO:teuthology.orchestra.run.smithi102.stdout:(70/113): python3-toml-0.10.2-6.el9.noarch.rpm 74 kB/s | 42 kB 00:00 2024-09-13T17:40:23.076 INFO:teuthology.orchestra.run.smithi102.stdout:(71/113): socat-1.7.4.1-6.el9.x86_64.rpm 361 kB/s | 304 kB 00:00 2024-09-13T17:40:23.410 INFO:teuthology.orchestra.run.smithi102.stdout:(72/113): xmlstarlet-1.6.1-20.el9.x86_64.rpm 98 kB/s | 64 kB 00:00 2024-09-13T17:40:23.602 INFO:teuthology.orchestra.run.smithi102.stdout:(73/113): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-13T17:40:23.644 INFO:teuthology.orchestra.run.smithi102.stdout:(74/113): gperftools-libs-2.9.1-3.el9.x86_64.rp 1.3 MB/s | 308 kB 00:00 2024-09-13T17:40:23.669 INFO:teuthology.orchestra.run.smithi102.stdout:(75/113): zlib-devel-1.2.11-41.el9.x86_64.rpm 76 kB/s | 45 kB 00:00 2024-09-13T17:40:23.695 INFO:teuthology.orchestra.run.smithi102.stdout:(76/113): liboath-2.6.7-2.el9.x86_64.rpm 530 kB/s | 49 kB 00:00 2024-09-13T17:40:23.728 INFO:teuthology.orchestra.run.smithi102.stdout:(77/113): libunwind-1.6.2-1.el9.x86_64.rpm 830 kB/s | 67 kB 00:00 2024-09-13T17:40:23.753 INFO:teuthology.orchestra.run.smithi102.stdout:(78/113): python3-cachetools-4.2.4-1.el9.noarch 618 kB/s | 32 kB 00:00 2024-09-13T17:40:23.778 INFO:teuthology.orchestra.run.smithi102.stdout:(79/113): python3-certifi-2023.05.07-4.el9.noar 289 kB/s | 14 kB 00:00 2024-09-13T17:40:23.803 INFO:teuthology.orchestra.run.smithi102.stdout:(80/113): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 324 kB/s | 43 kB 00:00 2024-09-13T17:40:23.829 INFO:teuthology.orchestra.run.smithi102.stdout:(81/113): python3-cheroot-8.6.0-4.el9.noarch.rp 2.2 MB/s | 172 kB 00:00 2024-09-13T17:40:23.862 INFO:teuthology.orchestra.run.smithi102.stdout:(82/113): python3-cherrypy-18.6.1-2.el9.noarch. 4.2 MB/s | 358 kB 00:00 2024-09-13T17:40:23.891 INFO:teuthology.orchestra.run.smithi102.stdout:(83/113): python3-jaraco-8.2.1-3.el9.noarch.rpm 181 kB/s | 11 kB 00:00 2024-09-13T17:40:23.930 INFO:teuthology.orchestra.run.smithi102.stdout:(84/113): python3-jaraco-classes-3.2.1-5.el9.no 262 kB/s | 18 kB 00:00 2024-09-13T17:40:23.955 INFO:teuthology.orchestra.run.smithi102.stdout:(85/113): python3-jaraco-collections-3.0.0-8.el 347 kB/s | 23 kB 00:00 2024-09-13T17:40:23.980 INFO:teuthology.orchestra.run.smithi102.stdout:(86/113): python3-jaraco-functools-3.5.0-2.el9. 389 kB/s | 19 kB 00:00 2024-09-13T17:40:24.006 INFO:teuthology.orchestra.run.smithi102.stdout:(87/113): python3-jaraco-text-3.2.0-6.el9.noarc 391 kB/s | 20 kB 00:00 2024-09-13T17:40:24.031 INFO:teuthology.orchestra.run.smithi102.stdout:(88/113): python3-jwt+crypto-2.4.0-1.el9.noarch 180 kB/s | 9.0 kB 00:00 2024-09-13T17:40:24.056 INFO:teuthology.orchestra.run.smithi102.stdout:(89/113): python3-google-auth-2.34.0-1.el9.noar 871 kB/s | 220 kB 00:00 2024-09-13T17:40:24.081 INFO:teuthology.orchestra.run.smithi102.stdout:(90/113): python3-jwt-2.4.0-1.el9.noarch.rpm 542 kB/s | 41 kB 00:00 2024-09-13T17:40:24.107 INFO:teuthology.orchestra.run.smithi102.stdout:(91/113): python3-logutils-0.3.5-21.el9.noarch. 922 kB/s | 46 kB 00:00 2024-09-13T17:40:24.132 INFO:teuthology.orchestra.run.smithi102.stdout:(92/113): python3-more-itertools-8.12.0-2.el9.n 1.5 MB/s | 79 kB 00:00 2024-09-13T17:40:24.157 INFO:teuthology.orchestra.run.smithi102.stdout:(93/113): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-09-13T17:40:24.165 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-13T17:40:24.182 INFO:teuthology.orchestra.run.smithi102.stdout:(94/113): python3-portend-3.1.0-2.el9.noarch.rp 328 kB/s | 16 kB 00:00 2024-09-13T17:40:24.216 INFO:teuthology.orchestra.run.smithi102.stdout:(95/113): python3-pyOpenSSL-21.0.0-1.el9.noarch 1.5 MB/s | 90 kB 00:00 2024-09-13T17:40:24.242 INFO:teuthology.orchestra.run.smithi102.stdout:(96/113): python3-repoze-lru-0.7-16.el9.noarch. 522 kB/s | 31 kB 00:00 2024-09-13T17:40:24.267 INFO:teuthology.orchestra.run.smithi102.stdout:(97/113): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-13T17:40:24.292 INFO:teuthology.orchestra.run.smithi102.stdout:(98/113): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-13T17:40:24.302 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/121 2024-09-13T17:40:24.317 INFO:teuthology.orchestra.run.smithi102.stdout:(99/113): python3-tempora-5.0.0-2.el9.noarch.rp 711 kB/s | 36 kB 00:00 2024-09-13T17:40:24.343 INFO:teuthology.orchestra.run.smithi102.stdout:(100/113): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-13T17:40:24.357 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/121 2024-09-13T17:40:24.376 INFO:teuthology.orchestra.run.smithi102.stdout:(101/113): python3-kubernetes-26.1.0-2.el9.noar 3.0 MB/s | 1.0 MB 00:00 2024-09-13T17:40:24.402 INFO:teuthology.orchestra.run.smithi102.stdout:(102/113): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-13T17:40:24.411 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/121 2024-09-13T17:40:24.428 INFO:teuthology.orchestra.run.smithi102.stdout:(103/113): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-09-13T17:40:24.446 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/121 2024-09-13T17:40:24.469 INFO:teuthology.orchestra.run.smithi102.stdout:(104/113): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-13T17:40:24.470 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/121 2024-09-13T17:40:24.525 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/121 2024-09-13T17:40:24.544 INFO:teuthology.orchestra.run.smithi102.stdout:(105/113): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-13T17:40:24.744 INFO:teuthology.orchestra.run.smithi102.stdout:(106/113): NetworkManager-1.49.90-1.el9.x86_64. 12 MB/s | 2.3 MB 00:00 2024-09-13T17:40:24.795 INFO:teuthology.orchestra.run.smithi102.stdout:(107/113): librbd1-17.2.7-1627.gbbdadfa3.el9.x8 9.0 MB/s | 2.9 MB 00:00 2024-09-13T17:40:24.827 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/121 2024-09-13T17:40:24.846 INFO:teuthology.orchestra.run.smithi102.stdout:(108/113): NetworkManager-libnm-1.49.90-1.el9.x 18 MB/s | 1.8 MB 00:00 2024-09-13T17:40:24.862 INFO:teuthology.orchestra.run.smithi102.stdout:(109/113): NetworkManager-team-1.49.90-1.el9.x8 611 kB/s | 41 kB 00:00 2024-09-13T17:40:24.887 INFO:teuthology.orchestra.run.smithi102.stdout:(110/113): NetworkManager-tui-1.49.90-1.el9.x86 5.8 MB/s | 249 kB 00:00 2024-09-13T17:40:24.947 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : librados2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 8/121 2024-09-13T17:40:24.962 INFO:teuthology.orchestra.run.smithi102.stdout:(111/113): librados2-17.2.7-1627.gbbdadfa3.el9. 5.8 MB/s | 3.1 MB 00:00 2024-09-13T17:40:25.021 INFO:teuthology.orchestra.run.smithi102.stdout:(112/113): openssl-libs-3.2.2-6.el9.x86_64.rpm 18 MB/s | 2.4 MB 00:00 2024-09-13T17:40:25.062 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 8/121 2024-09-13T17:40:25.073 INFO:teuthology.orchestra.run.smithi102.stdout:(113/113): openssl-3.2.2-6.el9.x86_64.rpm 6.5 MB/s | 1.4 MB 00:00 2024-09-13T17:40:25.078 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-13T17:40:25.078 INFO:teuthology.orchestra.run.smithi102.stdout:Total 12 MB/s | 196 MB 00:16 2024-09-13T17:40:25.107 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephfs2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 9/121 2024-09-13T17:40:25.585 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephfs2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 9/121 2024-09-13T17:40:25.617 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librgw2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 10/121 2024-09-13T17:40:25.709 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librgw2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 10/121 2024-09-13T17:40:25.749 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rados-2:17.2.7-1627.gbbdadfa3.el9.x86_64 11/121 2024-09-13T17:40:25.778 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephsqlite-2:17.2.7-1627.gbbdadfa3.el9.x86_64 12/121 2024-09-13T17:40:25.877 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1627.gbbdadfa3.el9.x86_64 12/121 2024-09-13T17:40:25.904 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libradosstriper1-2:17.2.7-1627.gbbdadfa3.el9.x86 13/121 2024-09-13T17:40:25.932 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-13T17:40:26.057 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-13T17:40:26.057 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-13T17:40:26.145 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1627.gbbdadfa3.el9.x86 13/121 2024-09-13T17:40:26.324 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 14/121 2024-09-13T17:40:26.456 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 15/121 2024-09-13T17:40:26.518 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 15/121 2024-09-13T17:40:26.807 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 15/121 2024-09-13T17:40:26.848 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 16/121 2024-09-13T17:40:26.913 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : liboath-2.6.7-2.el9.x86_64 17/121 2024-09-13T17:40:26.942 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 18/121 2024-09-13T17:40:27.009 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 19/121 2024-09-13T17:40:27.064 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 20/121 2024-09-13T17:40:27.095 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 21/121 2024-09-13T17:40:27.140 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 22/121 2024-09-13T17:40:27.182 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 23/121 2024-09-13T17:40:27.226 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ceph-argparse-2:17.2.7-1627.gbbdadfa3.el 24/121 2024-09-13T17:40:27.358 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cephfs-2:17.2.7-1627.gbbdadfa3.el9.x86_6 25/121 2024-09-13T17:40:27.418 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 26/121 2024-09-13T17:40:27.457 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rsa-4.9-2.el9.noarch 27/121 2024-09-13T17:40:27.509 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rgw-2:17.2.7-1627.gbbdadfa3.el9.x86_64 28/121 2024-09-13T17:40:27.547 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librados-devel-2:17.2.7-1627.gbbdadfa3.el9.x86_6 29/121 2024-09-13T17:40:27.589 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 30/121 2024-09-13T17:40:27.623 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 31/121 2024-09-13T17:40:27.686 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 32/121 2024-09-13T17:40:27.759 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 33/121 2024-09-13T17:40:27.793 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 34/121 2024-09-13T17:40:27.841 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 35/121 2024-09-13T17:40:27.885 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 36/121 2024-09-13T17:40:27.919 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 37/121 2024-09-13T17:40:27.969 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 38/121 2024-09-13T17:40:27.973 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-13T17:40:27.973 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-13T17:40:28.079 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 39/121 2024-09-13T17:40:28.120 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 40/121 2024-09-13T17:40:28.160 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 41/121 2024-09-13T17:40:28.195 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 42/121 2024-09-13T17:40:28.224 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 43/121 2024-09-13T17:40:28.260 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 44/121 2024-09-13T17:40:28.291 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 45/121 2024-09-13T17:40:28.342 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 46/121 2024-09-13T17:40:28.375 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 47/121 2024-09-13T17:40:28.416 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 48/121 2024-09-13T17:40:28.451 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 49/121 2024-09-13T17:40:28.486 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 50/121 2024-09-13T17:40:28.521 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 51/121 2024-09-13T17:40:28.568 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 52/121 2024-09-13T17:40:28.666 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-packaging-20.9-5.el9.noarch 53/121 2024-09-13T17:40:28.723 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 54/121 2024-09-13T17:40:29.454 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 55/121 2024-09-13T17:40:29.500 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 56/121 2024-09-13T17:40:29.571 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 57/121 2024-09-13T17:40:29.710 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 58/121 2024-09-13T17:40:29.768 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/121 2024-09-13T17:40:29.918 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 60/121 2024-09-13T17:40:30.209 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 61/121 2024-09-13T17:40:30.246 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 62/121 2024-09-13T17:40:30.442 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openblas-0.3.26-2.el9.x86_64 63/121 2024-09-13T17:40:30.468 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 64/121 2024-09-13T17:40:30.547 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 65/121 2024-09-13T17:40:31.201 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 66/121 2024-09-13T17:40:31.325 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 67/121 2024-09-13T17:40:32.587 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 68/121 2024-09-13T17:40:32.678 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 69/121 2024-09-13T17:40:32.705 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 70/121 2024-09-13T17:40:32.985 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-13T17:40:33.000 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 71/121 2024-09-13T17:40:33.043 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : librbd1-2:17.2.7-1627.gbbdadfa3.el9.x86_64 72/121 2024-09-13T17:40:33.110 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/121 2024-09-13T17:40:33.130 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librbd1-2:17.2.7-1627.gbbdadfa3.el9.x86_64 72/121 2024-09-13T17:40:33.171 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/121 2024-09-13T17:40:33.177 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rbd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 73/121 2024-09-13T17:40:33.226 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ply-3.11-14.el9.noarch 74/121 2024-09-13T17:40:33.235 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/121 2024-09-13T17:40:33.276 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/121 2024-09-13T17:40:33.281 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 75/121 2024-09-13T17:40:33.302 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/121 2024-09-13T17:40:33.363 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/121 2024-09-13T17:40:33.468 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 76/121 2024-09-13T17:40:33.517 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 77/121 2024-09-13T17:40:33.593 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 78/121 2024-09-13T17:40:33.662 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/121 2024-09-13T17:40:33.681 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 79/121 2024-09-13T17:40:33.721 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 80/121 2024-09-13T17:40:33.758 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 81/121 2024-09-13T17:40:33.761 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : librados2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 8/121 2024-09-13T17:40:33.793 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 82/121 2024-09-13T17:40:33.824 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 83/121 2024-09-13T17:40:33.854 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : fuse-2.9.9-16.el9.x86_64 84/121 2024-09-13T17:40:33.874 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 8/121 2024-09-13T17:40:33.884 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1627.gbbdadfa3.e 85/121 2024-09-13T17:40:33.904 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 9/121 2024-09-13T17:40:33.957 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1627.gbbdadfa3. 86/121 2024-09-13T17:40:33.988 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ceph-common-2:17.2.7-1627.gbbdadfa3.el9. 87/121 2024-09-13T17:40:34.385 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephfs2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 9/121 2024-09-13T17:40:34.416 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librgw2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 10/121 2024-09-13T17:40:34.507 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librgw2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 10/121 2024-09-13T17:40:34.549 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rados-2:17.2.7-1627.gbbdadfa3.el9.x86_64 11/121 2024-09-13T17:40:34.575 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephsqlite-2:17.2.7-1627.gbbdadfa3.el9.x86_64 12/121 2024-09-13T17:40:34.677 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1627.gbbdadfa3.el9.x86_64 12/121 2024-09-13T17:40:34.702 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libradosstriper1-2:17.2.7-1627.gbbdadfa3.el9.x86 13/121 2024-09-13T17:40:34.914 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1627.gbbdadfa3.el9.x86 13/121 2024-09-13T17:40:35.031 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 14/121 2024-09-13T17:40:35.156 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 15/121 2024-09-13T17:40:35.249 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 15/121 2024-09-13T17:40:35.532 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 15/121 2024-09-13T17:40:35.570 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 16/121 2024-09-13T17:40:35.635 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : liboath-2.6.7-2.el9.x86_64 17/121 2024-09-13T17:40:35.663 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 18/121 2024-09-13T17:40:35.732 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 19/121 2024-09-13T17:40:35.785 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 20/121 2024-09-13T17:40:35.825 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 21/121 2024-09-13T17:40:35.876 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 22/121 2024-09-13T17:40:35.912 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 23/121 2024-09-13T17:40:35.955 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ceph-argparse-2:17.2.7-1627.gbbdadfa3.el 24/121 2024-09-13T17:40:36.085 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cephfs-2:17.2.7-1627.gbbdadfa3.el9.x86_6 25/121 2024-09-13T17:40:36.148 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 26/121 2024-09-13T17:40:36.185 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rsa-4.9-2.el9.noarch 27/121 2024-09-13T17:40:36.237 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rgw-2:17.2.7-1627.gbbdadfa3.el9.x86_64 28/121 2024-09-13T17:40:36.275 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librados-devel-2:17.2.7-1627.gbbdadfa3.el9.x86_6 29/121 2024-09-13T17:40:36.317 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 30/121 2024-09-13T17:40:36.351 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 31/121 2024-09-13T17:40:36.379 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-common-2:17.2.7-1627.gbbdadfa3.el9.x86_64 88/121 2024-09-13T17:40:36.411 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-common-2:17.2.7-1627.gbbdadfa3.el9.x86_64 88/121 2024-09-13T17:40:36.414 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 32/121 2024-09-13T17:40:36.497 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 33/121 2024-09-13T17:40:36.530 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 34/121 2024-09-13T17:40:36.576 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 35/121 2024-09-13T17:40:36.625 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 36/121 2024-09-13T17:40:36.654 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 37/121 2024-09-13T17:40:36.695 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 38/121 2024-09-13T17:40:36.799 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 39/121 2024-09-13T17:40:36.839 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 40/121 2024-09-13T17:40:36.886 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 41/121 2024-09-13T17:40:36.926 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-common-2:17.2.7-1627.gbbdadfa3.el9.x86_64 88/121 2024-09-13T17:40:36.929 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 42/121 2024-09-13T17:40:36.968 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 43/121 2024-09-13T17:40:36.974 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-base-2:17.2.7-1627.gbbdadfa3.el9.x86_64 89/121 2024-09-13T17:40:37.003 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 44/121 2024-09-13T17:40:37.033 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 45/121 2024-09-13T17:40:37.063 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-base-2:17.2.7-1627.gbbdadfa3.el9.x86_64 89/121 2024-09-13T17:40:37.063 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-13T17:40:37.063 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-13T17:40:37.063 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:40:37.067 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 46/121 2024-09-13T17:40:37.109 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-selinux-2:17.2.7-1627.gbbdadfa3.el9.x86_64 90/121 2024-09-13T17:40:37.109 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 47/121 2024-09-13T17:40:37.150 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 48/121 2024-09-13T17:40:37.201 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 49/121 2024-09-13T17:40:37.237 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 50/121 2024-09-13T17:40:37.271 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 51/121 2024-09-13T17:40:37.326 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 52/121 2024-09-13T17:40:37.433 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-packaging-20.9-5.el9.noarch 53/121 2024-09-13T17:40:37.490 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 54/121 2024-09-13T17:40:38.202 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 55/121 2024-09-13T17:40:38.495 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 56/121 2024-09-13T17:40:38.599 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 57/121 2024-09-13T17:40:38.769 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 58/121 2024-09-13T17:40:38.823 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/121 2024-09-13T17:40:38.858 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 60/121 2024-09-13T17:40:38.959 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 61/121 2024-09-13T17:40:39.008 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 62/121 2024-09-13T17:40:39.201 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openblas-0.3.26-2.el9.x86_64 63/121 2024-09-13T17:40:39.224 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 64/121 2024-09-13T17:40:39.301 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 65/121 2024-09-13T17:40:39.955 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 66/121 2024-09-13T17:40:40.128 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 67/121 2024-09-13T17:40:41.379 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 68/121 2024-09-13T17:40:41.463 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 69/121 2024-09-13T17:40:41.490 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 70/121 2024-09-13T17:40:41.781 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 71/121 2024-09-13T17:40:41.828 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : librbd1-2:17.2.7-1627.gbbdadfa3.el9.x86_64 72/121 2024-09-13T17:40:41.918 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-2:17.2.7-1627.gbbdadfa3.el9.x86_64 72/121 2024-09-13T17:40:41.958 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rbd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 73/121 2024-09-13T17:40:42.023 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ply-3.11-14.el9.noarch 74/121 2024-09-13T17:40:42.079 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 75/121 2024-09-13T17:40:42.276 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 76/121 2024-09-13T17:40:42.320 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 77/121 2024-09-13T17:40:42.398 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 78/121 2024-09-13T17:40:42.487 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 79/121 2024-09-13T17:40:42.541 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 80/121 2024-09-13T17:40:42.578 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 81/121 2024-09-13T17:40:42.612 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 82/121 2024-09-13T17:40:42.644 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 83/121 2024-09-13T17:40:42.673 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : fuse-2.9.9-16.el9.x86_64 84/121 2024-09-13T17:40:42.704 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1627.gbbdadfa3.e 85/121 2024-09-13T17:40:42.776 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1627.gbbdadfa3. 86/121 2024-09-13T17:40:42.806 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ceph-common-2:17.2.7-1627.gbbdadfa3.el9. 87/121 2024-09-13T17:40:45.196 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-2:17.2.7-1627.gbbdadfa3.el9.x86_64 88/121 2024-09-13T17:40:45.246 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-common-2:17.2.7-1627.gbbdadfa3.el9.x86_64 88/121 2024-09-13T17:40:45.760 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-2:17.2.7-1627.gbbdadfa3.el9.x86_64 88/121 2024-09-13T17:40:45.800 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-base-2:17.2.7-1627.gbbdadfa3.el9.x86_64 89/121 2024-09-13T17:40:45.905 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-base-2:17.2.7-1627.gbbdadfa3.el9.x86_64 89/121 2024-09-13T17:40:45.905 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-13T17:40:45.905 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-13T17:40:45.905 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:40:45.943 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-selinux-2:17.2.7-1627.gbbdadfa3.el9.x86_64 90/121 2024-09-13T17:40:48.310 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1627.gbbdadfa3.el9.x86_64 90/121 2024-09-13T17:40:48.310 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /sys 2024-09-13T17:40:48.310 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /proc 2024-09-13T17:40:48.310 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /mnt 2024-09-13T17:40:48.310 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /var/tmp 2024-09-13T17:40:48.311 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /home 2024-09-13T17:40:48.311 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /root 2024-09-13T17:40:48.311 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /tmp 2024-09-13T17:40:48.311 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:40:49.374 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1627.gbbdadfa3.el9.n 91/121 2024-09-13T17:40:50.347 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1627.gbbdadfa3.el9.n 91/121 2024-09-13T17:40:50.391 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1627.gbbd 92/121 2024-09-13T17:40:50.493 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1627.gbbd 92/121 2024-09-13T17:40:50.646 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1627.gbbdadfa3.el 93/121 2024-09-13T17:40:50.685 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-2:17.2.7-1627.gbbdadfa3.el9.x86_64 94/121 2024-09-13T17:40:50.709 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1627.gbbdadfa3.el9.x86_64 94/121 2024-09-13T17:40:50.709 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:40:50.709 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-13T17:40:50.709 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T17:40:50.710 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T17:40:50.710 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:40:50.753 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-rook-2:17.2.7-1627.gbbdadfa3.el9.noarch 95/121 2024-09-13T17:40:50.935 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1627.gbbdadfa3.el9.noarch 95/121 2024-09-13T17:40:50.962 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mds-2:17.2.7-1627.gbbdadfa3.el9.x86_64 96/121 2024-09-13T17:40:50.985 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mds-2:17.2.7-1627.gbbdadfa3.el9.x86_64 96/121 2024-09-13T17:40:50.985 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:40:50.986 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-13T17:40:50.986 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T17:40:50.986 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T17:40:50.986 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:40:51.354 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mon-2:17.2.7-1627.gbbdadfa3.el9.x86_64 97/121 2024-09-13T17:40:51.378 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mon-2:17.2.7-1627.gbbdadfa3.el9.x86_64 97/121 2024-09-13T17:40:51.378 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:40:51.378 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-13T17:40:51.378 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T17:40:51.378 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T17:40:51.378 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:40:52.690 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-osd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 98/121 2024-09-13T17:40:52.716 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-osd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 98/121 2024-09-13T17:40:52.716 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:40:52.716 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-13T17:40:52.716 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T17:40:52.716 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T17:40:52.716 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:40:53.977 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-2:17.2.7-1627.gbbdadfa3.el9.x86_64 99/121 2024-09-13T17:40:54.008 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-radosgw-2:17.2.7-1627.gbbdadfa3.el9.x86_64 100/121 2024-09-13T17:40:54.087 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1627.gbbdadfa3.el9.x86_64 100/121 2024-09-13T17:40:54.087 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:40:54.087 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-13T17:40:54.087 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T17:40:54.087 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T17:40:54.087 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:40:54.384 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-mirror-2:17.2.7-1627.gbbdadfa3.el9.x86_64 101/121 2024-09-13T17:40:54.407 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1627.gbbdadfa3.el9.x86_64 101/121 2024-09-13T17:40:54.408 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:40:54.408 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-13T17:40:54.408 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T17:40:54.408 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T17:40:54.408 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:40:57.114 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1627.gbbdadfa3.el9.x86_64 90/121 2024-09-13T17:40:57.114 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /sys 2024-09-13T17:40:57.114 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /proc 2024-09-13T17:40:57.114 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /mnt 2024-09-13T17:40:57.114 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /var/tmp 2024-09-13T17:40:57.114 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /home 2024-09-13T17:40:57.114 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /root 2024-09-13T17:40:57.114 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /tmp 2024-09-13T17:40:57.114 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:40:58.137 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1627.gbbdadfa3.el9.n 91/121 2024-09-13T17:40:58.859 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-test-2:17.2.7-1627.gbbdadfa3.el9.x86_64 102/121 2024-09-13T17:40:58.906 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-fuse-2:17.2.7-1627.gbbdadfa3.el9.x86_64 103/121 2024-09-13T17:40:58.957 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-fuse-2:17.2.7-1627.gbbdadfa3.el9.x86_64 104/121 2024-09-13T17:40:58.994 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-nbd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 105/121 2024-09-13T17:40:59.020 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 106/121 2024-09-13T17:40:59.044 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephfs-devel-2:17.2.7-1627.gbbdadfa3.el9.x86_ 107/121 2024-09-13T17:40:59.073 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 108/121 2024-09-13T17:40:59.146 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1627.gbbdadfa3.el9.n 91/121 2024-09-13T17:40:59.182 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1627.gbbd 92/121 2024-09-13T17:40:59.284 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1627.gbbd 92/121 2024-09-13T17:40:59.444 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1627.gbbdadfa3.el 93/121 2024-09-13T17:40:59.476 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-2:17.2.7-1627.gbbdadfa3.el9.x86_64 94/121 2024-09-13T17:40:59.500 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1627.gbbdadfa3.el9.x86_64 94/121 2024-09-13T17:40:59.500 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:40:59.500 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-13T17:40:59.500 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T17:40:59.501 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T17:40:59.501 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:40:59.544 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-rook-2:17.2.7-1627.gbbdadfa3.el9.noarch 95/121 2024-09-13T17:40:59.729 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1627.gbbdadfa3.el9.noarch 95/121 2024-09-13T17:40:59.761 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mds-2:17.2.7-1627.gbbdadfa3.el9.x86_64 96/121 2024-09-13T17:40:59.784 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mds-2:17.2.7-1627.gbbdadfa3.el9.x86_64 96/121 2024-09-13T17:40:59.784 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:40:59.784 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-13T17:40:59.784 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T17:40:59.785 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T17:40:59.785 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:40:59.922 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 109/121 2024-09-13T17:41:00.146 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mon-2:17.2.7-1627.gbbdadfa3.el9.x86_64 97/121 2024-09-13T17:41:00.170 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mon-2:17.2.7-1627.gbbdadfa3.el9.x86_64 97/121 2024-09-13T17:41:00.170 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:41:00.171 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-13T17:41:00.171 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T17:41:00.171 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T17:41:00.171 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:00.188 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 110/121 2024-09-13T17:41:00.283 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 111/121 2024-09-13T17:41:00.376 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flex-2.6.4-9.el9.x86_64 112/121 2024-09-13T17:41:00.410 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : bison-3.7.4-5.el9.x86_64 113/121 2024-09-13T17:41:00.436 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 114/121 2024-09-13T17:41:00.470 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 115/121 2024-09-13T17:41:00.470 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 116/121 2024-09-13T17:41:00.554 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 116/121 2024-09-13T17:41:00.554 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 117/121 2024-09-13T17:41:00.639 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 117/121 2024-09-13T17:41:00.663 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 118/121 2024-09-13T17:41:00.669 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 119/121 2024-09-13T17:41:00.669 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 119/121 2024-09-13T17:41:00.858 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 119/121 2024-09-13T17:41:00.858 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 120/121 2024-09-13T17:41:00.935 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 120/121 2024-09-13T17:41:01.007 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 121/121 2024-09-13T17:41:01.479 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-osd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 98/121 2024-09-13T17:41:01.507 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-osd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 98/121 2024-09-13T17:41:01.507 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:41:01.507 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-13T17:41:01.507 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T17:41:01.507 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T17:41:01.508 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:02.763 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-2:17.2.7-1627.gbbdadfa3.el9.x86_64 99/121 2024-09-13T17:41:02.797 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-radosgw-2:17.2.7-1627.gbbdadfa3.el9.x86_64 100/121 2024-09-13T17:41:02.874 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1627.gbbdadfa3.el9.x86_64 100/121 2024-09-13T17:41:02.875 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:41:02.875 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-13T17:41:02.875 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T17:41:02.875 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T17:41:02.875 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:03.165 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-mirror-2:17.2.7-1627.gbbdadfa3.el9.x86_64 101/121 2024-09-13T17:41:03.189 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1627.gbbdadfa3.el9.x86_64 101/121 2024-09-13T17:41:03.189 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T17:41:03.189 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-13T17:41:03.189 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T17:41:03.189 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T17:41:03.189 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:06.107 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 121/121 2024-09-13T17:41:06.107 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-2:17.2.7-1627.gbbdadfa3.el9.x86_64 1/121 2024-09-13T17:41:06.107 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-base-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2/121 2024-09-13T17:41:06.108 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-common-2:17.2.7-1627.gbbdadfa3.el9.x86_64 3/121 2024-09-13T17:41:06.108 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-fuse-2:17.2.7-1627.gbbdadfa3.el9.x86_64 4/121 2024-09-13T17:41:06.108 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mds-2:17.2.7-1627.gbbdadfa3.el9.x86_64 5/121 2024-09-13T17:41:06.108 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-2:17.2.7-1627.gbbdadfa3.el9.x86_64 6/121 2024-09-13T17:41:06.108 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mon-2:17.2.7-1627.gbbdadfa3.el9.x86_64 7/121 2024-09-13T17:41:06.108 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-osd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 8/121 2024-09-13T17:41:06.108 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-radosgw-2:17.2.7-1627.gbbdadfa3.el9.x86_64 9/121 2024-09-13T17:41:06.108 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-selinux-2:17.2.7-1627.gbbdadfa3.el9.x86_64 10/121 2024-09-13T17:41:06.108 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-test-2:17.2.7-1627.gbbdadfa3.el9.x86_64 11/121 2024-09-13T17:41:06.108 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephfs-devel-2:17.2.7-1627.gbbdadfa3.el9.x86_ 12/121 2024-09-13T17:41:06.108 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephfs2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 13/121 2024-09-13T17:41:06.108 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephsqlite-2:17.2.7-1627.gbbdadfa3.el9.x86_64 14/121 2024-09-13T17:41:06.108 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados-devel-2:17.2.7-1627.gbbdadfa3.el9.x86_6 15/121 2024-09-13T17:41:06.109 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libradosstriper1-2:17.2.7-1627.gbbdadfa3.el9.x86 16/121 2024-09-13T17:41:06.109 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librgw2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 17/121 2024-09-13T17:41:06.109 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1627.gbbdadfa3.el 18/121 2024-09-13T17:41:06.109 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ceph-common-2:17.2.7-1627.gbbdadfa3.el9. 19/121 2024-09-13T17:41:06.109 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cephfs-2:17.2.7-1627.gbbdadfa3.el9.x86_6 20/121 2024-09-13T17:41:06.109 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rados-2:17.2.7-1627.gbbdadfa3.el9.x86_64 21/121 2024-09-13T17:41:06.109 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rbd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 22/121 2024-09-13T17:41:06.109 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rgw-2:17.2.7-1627.gbbdadfa3.el9.x86_64 23/121 2024-09-13T17:41:06.109 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-fuse-2:17.2.7-1627.gbbdadfa3.el9.x86_64 24/121 2024-09-13T17:41:06.109 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-mirror-2:17.2.7-1627.gbbdadfa3.el9.x86_64 25/121 2024-09-13T17:41:06.109 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-nbd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 26/121 2024-09-13T17:41:06.109 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1627.gbbdadfa3. 27/121 2024-09-13T17:41:06.118 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1627.gbbdadfa3.el9.n 28/121 2024-09-13T17:41:06.118 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1627.gbbd 29/121 2024-09-13T17:41:06.118 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1627.gbbdadfa3.el 30/121 2024-09-13T17:41:06.118 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1627.gbbdadfa3.el9.noarch 31/121 2024-09-13T17:41:06.118 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1627.gbbdadfa3.e 32/121 2024-09-13T17:41:06.118 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 33/121 2024-09-13T17:41:06.118 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 34/121 2024-09-13T17:41:06.118 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 35/121 2024-09-13T17:41:06.118 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 36/121 2024-09-13T17:41:06.119 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 37/121 2024-09-13T17:41:06.119 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ply-3.11-14.el9.noarch 38/121 2024-09-13T17:41:06.119 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 39/121 2024-09-13T17:41:06.119 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 40/121 2024-09-13T17:41:06.119 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 41/121 2024-09-13T17:41:06.119 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : bison-3.7.4-5.el9.x86_64 42/121 2024-09-13T17:41:06.119 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 43/121 2024-09-13T17:41:06.119 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/121 2024-09-13T17:41:06.119 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/121 2024-09-13T17:41:06.119 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/121 2024-09-13T17:41:06.119 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/121 2024-09-13T17:41:06.119 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/121 2024-09-13T17:41:06.119 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/121 2024-09-13T17:41:06.120 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/121 2024-09-13T17:41:06.120 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 51/121 2024-09-13T17:41:06.120 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 52/121 2024-09-13T17:41:06.120 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 53/121 2024-09-13T17:41:06.120 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 54/121 2024-09-13T17:41:06.120 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 55/121 2024-09-13T17:41:06.120 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 56/121 2024-09-13T17:41:06.120 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 57/121 2024-09-13T17:41:06.120 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 58/121 2024-09-13T17:41:06.121 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 59/121 2024-09-13T17:41:06.121 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 60/121 2024-09-13T17:41:06.121 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 61/121 2024-09-13T17:41:06.121 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 62/121 2024-09-13T17:41:06.121 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 63/121 2024-09-13T17:41:06.121 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 64/121 2024-09-13T17:41:06.121 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 65/121 2024-09-13T17:41:06.121 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 66/121 2024-09-13T17:41:06.121 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 67/121 2024-09-13T17:41:06.121 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 68/121 2024-09-13T17:41:06.121 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 69/121 2024-09-13T17:41:06.121 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 70/121 2024-09-13T17:41:06.122 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 71/121 2024-09-13T17:41:06.122 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 72/121 2024-09-13T17:41:06.122 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 73/121 2024-09-13T17:41:06.122 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 74/121 2024-09-13T17:41:06.122 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 75/121 2024-09-13T17:41:06.122 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 76/121 2024-09-13T17:41:06.122 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 77/121 2024-09-13T17:41:06.123 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 78/121 2024-09-13T17:41:06.123 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 79/121 2024-09-13T17:41:06.123 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 80/121 2024-09-13T17:41:06.123 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 81/121 2024-09-13T17:41:06.123 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 82/121 2024-09-13T17:41:06.123 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 83/121 2024-09-13T17:41:06.123 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 84/121 2024-09-13T17:41:06.123 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 85/121 2024-09-13T17:41:06.123 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 86/121 2024-09-13T17:41:06.123 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 87/121 2024-09-13T17:41:06.123 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 88/121 2024-09-13T17:41:06.123 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 89/121 2024-09-13T17:41:06.123 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 90/121 2024-09-13T17:41:06.124 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 91/121 2024-09-13T17:41:06.124 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 92/121 2024-09-13T17:41:06.124 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 93/121 2024-09-13T17:41:06.124 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 94/121 2024-09-13T17:41:06.124 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 95/121 2024-09-13T17:41:06.124 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 96/121 2024-09-13T17:41:06.124 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 97/121 2024-09-13T17:41:06.124 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 98/121 2024-09-13T17:41:06.124 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 99/121 2024-09-13T17:41:06.124 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 100/121 2024-09-13T17:41:06.124 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 101/121 2024-09-13T17:41:06.124 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 102/121 2024-09-13T17:41:06.124 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 103/121 2024-09-13T17:41:06.124 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 104/121 2024-09-13T17:41:06.125 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 105/121 2024-09-13T17:41:06.125 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 106/121 2024-09-13T17:41:06.125 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 107/121 2024-09-13T17:41:06.125 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librbd1-2:17.2.7-1627.gbbdadfa3.el9.x86_64 108/121 2024-09-13T17:41:06.125 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 109/121 2024-09-13T17:41:06.125 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 110/121 2024-09-13T17:41:06.125 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 111/121 2024-09-13T17:41:06.125 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 112/121 2024-09-13T17:41:06.125 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 113/121 2024-09-13T17:41:06.125 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 114/121 2024-09-13T17:41:06.125 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 115/121 2024-09-13T17:41:06.125 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 116/121 2024-09-13T17:41:06.125 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 117/121 2024-09-13T17:41:06.126 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 118/121 2024-09-13T17:41:06.126 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 119/121 2024-09-13T17:41:06.126 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 120/121 2024-09-13T17:41:07.630 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-test-2:17.2.7-1627.gbbdadfa3.el9.x86_64 102/121 2024-09-13T17:41:07.662 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-fuse-2:17.2.7-1627.gbbdadfa3.el9.x86_64 103/121 2024-09-13T17:41:07.702 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-fuse-2:17.2.7-1627.gbbdadfa3.el9.x86_64 104/121 2024-09-13T17:41:07.742 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-nbd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 105/121 2024-09-13T17:41:07.767 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 106/121 2024-09-13T17:41:07.789 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs-devel-2:17.2.7-1627.gbbdadfa3.el9.x86_ 107/121 2024-09-13T17:41:07.819 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 108/121 2024-09-13T17:41:08.164 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 121/121 2024-09-13T17:41:08.164 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:41:08.164 INFO:teuthology.orchestra.run.smithi171.stdout:Upgraded: 2024-09-13T17:41:08.164 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-13T17:41:08.164 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-13T17:41:08.164 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-13T17:41:08.164 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-13T17:41:08.164 INFO:teuthology.orchestra.run.smithi171.stdout: librados2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:08.164 INFO:teuthology.orchestra.run.smithi171.stdout: librbd1-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:08.164 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-13T17:41:08.164 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-13T17:41:08.165 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-13T17:41:08.165 INFO:teuthology.orchestra.run.smithi171.stdout: bison-3.7.4-5.el9.x86_64 2024-09-13T17:41:08.165 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:08.165 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-base-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:08.165 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-common-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:08.165 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-fuse-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:08.165 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-grafana-dashboards-2:17.2.7-1627.gbbdadfa3.el9.noarch 2024-09-13T17:41:08.165 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mds-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:08.165 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:08.165 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-dashboard-2:17.2.7-1627.gbbdadfa3.el9.noarch 2024-09-13T17:41:08.165 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1627.gbbdadfa3.el9.noarch 2024-09-13T17:41:08.165 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-modules-core-2:17.2.7-1627.gbbdadfa3.el9.noarch 2024-09-13T17:41:08.165 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-rook-2:17.2.7-1627.gbbdadfa3.el9.noarch 2024-09-13T17:41:08.166 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mon-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:08.166 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-osd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:08.166 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-prometheus-alerts-2:17.2.7-1627.gbbdadfa3.el9.noarch 2024-09-13T17:41:08.166 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-radosgw-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:08.166 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-selinux-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:08.166 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-test-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:08.166 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-13T17:41:08.166 INFO:teuthology.orchestra.run.smithi171.stdout: flex-2.6.4-9.el9.x86_64 2024-09-13T17:41:08.166 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-13T17:41:08.166 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-13T17:41:08.166 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-13T17:41:08.166 INFO:teuthology.orchestra.run.smithi171.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-13T17:41:08.166 INFO:teuthology.orchestra.run.smithi171.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-13T17:41:08.166 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs-devel-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:08.167 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:08.167 INFO:teuthology.orchestra.run.smithi171.stdout: libcephsqlite-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:08.167 INFO:teuthology.orchestra.run.smithi171.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-13T17:41:08.167 INFO:teuthology.orchestra.run.smithi171.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-13T17:41:08.167 INFO:teuthology.orchestra.run.smithi171.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-13T17:41:08.167 INFO:teuthology.orchestra.run.smithi171.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-13T17:41:08.167 INFO:teuthology.orchestra.run.smithi171.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-13T17:41:08.167 INFO:teuthology.orchestra.run.smithi171.stdout: librados-devel-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:08.167 INFO:teuthology.orchestra.run.smithi171.stdout: libradosstriper1-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:08.167 INFO:teuthology.orchestra.run.smithi171.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-13T17:41:08.167 INFO:teuthology.orchestra.run.smithi171.stdout: librgw2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:08.167 INFO:teuthology.orchestra.run.smithi171.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-13T17:41:08.167 INFO:teuthology.orchestra.run.smithi171.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-13T17:41:08.168 INFO:teuthology.orchestra.run.smithi171.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-13T17:41:08.168 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-13T17:41:08.168 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-13T17:41:08.168 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-13T17:41:08.168 INFO:teuthology.orchestra.run.smithi171.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-13T17:41:08.168 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-13T17:41:08.168 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-argparse-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:08.168 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-common-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:08.168 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cephfs-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:08.168 INFO:teuthology.orchestra.run.smithi171.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-13T17:41:08.168 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-13T17:41:08.168 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-13T17:41:08.168 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-13T17:41:08.168 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-13T17:41:08.169 INFO:teuthology.orchestra.run.smithi171.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-13T17:41:08.169 INFO:teuthology.orchestra.run.smithi171.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-13T17:41:08.169 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-13T17:41:08.169 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-13T17:41:08.169 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-13T17:41:08.169 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-13T17:41:08.169 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-13T17:41:08.169 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-13T17:41:08.169 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-13T17:41:08.169 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-13T17:41:08.169 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-13T17:41:08.169 INFO:teuthology.orchestra.run.smithi171.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-13T17:41:08.169 INFO:teuthology.orchestra.run.smithi171.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-13T17:41:08.169 INFO:teuthology.orchestra.run.smithi171.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-13T17:41:08.170 INFO:teuthology.orchestra.run.smithi171.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-13T17:41:08.170 INFO:teuthology.orchestra.run.smithi171.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-13T17:41:08.170 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-13T17:41:08.170 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-13T17:41:08.170 INFO:teuthology.orchestra.run.smithi171.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-13T17:41:08.170 INFO:teuthology.orchestra.run.smithi171.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-13T17:41:08.170 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-13T17:41:08.170 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ply-3.11-14.el9.noarch 2024-09-13T17:41:08.170 INFO:teuthology.orchestra.run.smithi171.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-13T17:41:08.170 INFO:teuthology.orchestra.run.smithi171.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-13T17:41:08.170 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-13T17:41:08.170 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-13T17:41:08.170 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-13T17:41:08.170 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-13T17:41:08.171 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-13T17:41:08.171 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rados-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:08.171 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rbd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:08.171 INFO:teuthology.orchestra.run.smithi171.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-13T17:41:08.171 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-13T17:41:08.171 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-13T17:41:08.171 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rgw-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:08.171 INFO:teuthology.orchestra.run.smithi171.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-13T17:41:08.171 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-13T17:41:08.171 INFO:teuthology.orchestra.run.smithi171.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-13T17:41:08.171 INFO:teuthology.orchestra.run.smithi171.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-13T17:41:08.171 INFO:teuthology.orchestra.run.smithi171.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-13T17:41:08.171 INFO:teuthology.orchestra.run.smithi171.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-13T17:41:08.172 INFO:teuthology.orchestra.run.smithi171.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-13T17:41:08.172 INFO:teuthology.orchestra.run.smithi171.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-13T17:41:08.172 INFO:teuthology.orchestra.run.smithi171.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-13T17:41:08.172 INFO:teuthology.orchestra.run.smithi171.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-13T17:41:08.172 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-fuse-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:08.172 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-mirror-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:08.172 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-nbd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:08.172 INFO:teuthology.orchestra.run.smithi171.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-13T17:41:08.172 INFO:teuthology.orchestra.run.smithi171.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-13T17:41:08.172 INFO:teuthology.orchestra.run.smithi171.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-13T17:41:08.173 INFO:teuthology.orchestra.run.smithi171.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-13T17:41:08.173 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:41:08.173 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-13T17:41:08.573 DEBUG:teuthology.parallel:result is None 2024-09-13T17:41:08.670 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 109/121 2024-09-13T17:41:08.885 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 110/121 2024-09-13T17:41:08.990 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 111/121 2024-09-13T17:41:09.085 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flex-2.6.4-9.el9.x86_64 112/121 2024-09-13T17:41:09.116 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : bison-3.7.4-5.el9.x86_64 113/121 2024-09-13T17:41:09.142 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 114/121 2024-09-13T17:41:09.184 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 115/121 2024-09-13T17:41:09.185 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 116/121 2024-09-13T17:41:09.277 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 116/121 2024-09-13T17:41:09.277 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 117/121 2024-09-13T17:41:09.386 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 117/121 2024-09-13T17:41:09.411 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 118/121 2024-09-13T17:41:09.417 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 119/121 2024-09-13T17:41:09.417 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 119/121 2024-09-13T17:41:09.596 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 119/121 2024-09-13T17:41:09.596 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 120/121 2024-09-13T17:41:09.681 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 120/121 2024-09-13T17:41:09.752 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 121/121 2024-09-13T17:41:14.818 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 121/121 2024-09-13T17:41:14.818 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-2:17.2.7-1627.gbbdadfa3.el9.x86_64 1/121 2024-09-13T17:41:14.818 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-base-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2/121 2024-09-13T17:41:14.818 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-common-2:17.2.7-1627.gbbdadfa3.el9.x86_64 3/121 2024-09-13T17:41:14.819 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-fuse-2:17.2.7-1627.gbbdadfa3.el9.x86_64 4/121 2024-09-13T17:41:14.819 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mds-2:17.2.7-1627.gbbdadfa3.el9.x86_64 5/121 2024-09-13T17:41:14.819 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-2:17.2.7-1627.gbbdadfa3.el9.x86_64 6/121 2024-09-13T17:41:14.819 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mon-2:17.2.7-1627.gbbdadfa3.el9.x86_64 7/121 2024-09-13T17:41:14.819 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-osd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 8/121 2024-09-13T17:41:14.819 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-radosgw-2:17.2.7-1627.gbbdadfa3.el9.x86_64 9/121 2024-09-13T17:41:14.819 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-selinux-2:17.2.7-1627.gbbdadfa3.el9.x86_64 10/121 2024-09-13T17:41:14.819 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-test-2:17.2.7-1627.gbbdadfa3.el9.x86_64 11/121 2024-09-13T17:41:14.820 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs-devel-2:17.2.7-1627.gbbdadfa3.el9.x86_ 12/121 2024-09-13T17:41:14.820 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 13/121 2024-09-13T17:41:14.820 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephsqlite-2:17.2.7-1627.gbbdadfa3.el9.x86_64 14/121 2024-09-13T17:41:14.820 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados-devel-2:17.2.7-1627.gbbdadfa3.el9.x86_6 15/121 2024-09-13T17:41:14.820 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libradosstriper1-2:17.2.7-1627.gbbdadfa3.el9.x86 16/121 2024-09-13T17:41:14.820 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librgw2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 17/121 2024-09-13T17:41:14.820 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1627.gbbdadfa3.el 18/121 2024-09-13T17:41:14.820 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ceph-common-2:17.2.7-1627.gbbdadfa3.el9. 19/121 2024-09-13T17:41:14.821 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cephfs-2:17.2.7-1627.gbbdadfa3.el9.x86_6 20/121 2024-09-13T17:41:14.821 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rados-2:17.2.7-1627.gbbdadfa3.el9.x86_64 21/121 2024-09-13T17:41:14.821 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rbd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 22/121 2024-09-13T17:41:14.821 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rgw-2:17.2.7-1627.gbbdadfa3.el9.x86_64 23/121 2024-09-13T17:41:14.821 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-fuse-2:17.2.7-1627.gbbdadfa3.el9.x86_64 24/121 2024-09-13T17:41:14.821 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-mirror-2:17.2.7-1627.gbbdadfa3.el9.x86_64 25/121 2024-09-13T17:41:14.821 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-nbd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 26/121 2024-09-13T17:41:14.821 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1627.gbbdadfa3. 27/121 2024-09-13T17:41:14.821 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1627.gbbdadfa3.el9.n 28/121 2024-09-13T17:41:14.822 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1627.gbbd 29/121 2024-09-13T17:41:14.822 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1627.gbbdadfa3.el 30/121 2024-09-13T17:41:14.822 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1627.gbbdadfa3.el9.noarch 31/121 2024-09-13T17:41:14.822 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1627.gbbdadfa3.e 32/121 2024-09-13T17:41:14.822 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 33/121 2024-09-13T17:41:14.822 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 34/121 2024-09-13T17:41:14.822 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 35/121 2024-09-13T17:41:14.823 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 36/121 2024-09-13T17:41:14.823 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 37/121 2024-09-13T17:41:14.823 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ply-3.11-14.el9.noarch 38/121 2024-09-13T17:41:14.823 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 39/121 2024-09-13T17:41:14.823 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 40/121 2024-09-13T17:41:14.823 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 41/121 2024-09-13T17:41:14.823 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : bison-3.7.4-5.el9.x86_64 42/121 2024-09-13T17:41:14.823 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 43/121 2024-09-13T17:41:14.823 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/121 2024-09-13T17:41:14.824 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/121 2024-09-13T17:41:14.824 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/121 2024-09-13T17:41:14.824 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/121 2024-09-13T17:41:14.824 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/121 2024-09-13T17:41:14.824 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/121 2024-09-13T17:41:14.824 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/121 2024-09-13T17:41:14.824 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 51/121 2024-09-13T17:41:14.824 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 52/121 2024-09-13T17:41:14.824 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 53/121 2024-09-13T17:41:14.824 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 54/121 2024-09-13T17:41:14.824 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 55/121 2024-09-13T17:41:14.825 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 56/121 2024-09-13T17:41:14.825 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 57/121 2024-09-13T17:41:14.825 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 58/121 2024-09-13T17:41:14.825 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 59/121 2024-09-13T17:41:14.825 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 60/121 2024-09-13T17:41:14.825 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 61/121 2024-09-13T17:41:14.825 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 62/121 2024-09-13T17:41:14.825 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 63/121 2024-09-13T17:41:14.826 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 64/121 2024-09-13T17:41:14.826 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 65/121 2024-09-13T17:41:14.826 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 66/121 2024-09-13T17:41:14.826 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 67/121 2024-09-13T17:41:14.826 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 68/121 2024-09-13T17:41:14.826 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 69/121 2024-09-13T17:41:14.826 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 70/121 2024-09-13T17:41:14.826 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 71/121 2024-09-13T17:41:14.826 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 72/121 2024-09-13T17:41:14.826 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 73/121 2024-09-13T17:41:14.827 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 74/121 2024-09-13T17:41:14.827 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 75/121 2024-09-13T17:41:14.827 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 76/121 2024-09-13T17:41:14.827 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 77/121 2024-09-13T17:41:14.827 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 78/121 2024-09-13T17:41:14.827 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 79/121 2024-09-13T17:41:14.827 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 80/121 2024-09-13T17:41:14.827 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 81/121 2024-09-13T17:41:14.828 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 82/121 2024-09-13T17:41:14.828 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 83/121 2024-09-13T17:41:14.828 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 84/121 2024-09-13T17:41:14.828 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 85/121 2024-09-13T17:41:14.828 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 86/121 2024-09-13T17:41:14.828 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 87/121 2024-09-13T17:41:14.828 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 88/121 2024-09-13T17:41:14.828 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 89/121 2024-09-13T17:41:14.828 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 90/121 2024-09-13T17:41:14.828 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 91/121 2024-09-13T17:41:14.829 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 92/121 2024-09-13T17:41:14.829 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 93/121 2024-09-13T17:41:14.829 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 94/121 2024-09-13T17:41:14.829 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 95/121 2024-09-13T17:41:14.829 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 96/121 2024-09-13T17:41:14.829 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 97/121 2024-09-13T17:41:14.829 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 98/121 2024-09-13T17:41:14.829 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 99/121 2024-09-13T17:41:14.830 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 100/121 2024-09-13T17:41:14.830 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 101/121 2024-09-13T17:41:14.830 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 102/121 2024-09-13T17:41:14.830 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 103/121 2024-09-13T17:41:14.830 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 104/121 2024-09-13T17:41:14.830 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 105/121 2024-09-13T17:41:14.830 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 106/121 2024-09-13T17:41:14.830 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 107/121 2024-09-13T17:41:14.830 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-2:17.2.7-1627.gbbdadfa3.el9.x86_64 108/121 2024-09-13T17:41:14.830 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 109/121 2024-09-13T17:41:14.831 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 110/121 2024-09-13T17:41:14.831 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 111/121 2024-09-13T17:41:14.831 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 112/121 2024-09-13T17:41:14.831 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 113/121 2024-09-13T17:41:14.831 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 114/121 2024-09-13T17:41:14.831 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 115/121 2024-09-13T17:41:14.831 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 116/121 2024-09-13T17:41:14.831 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 117/121 2024-09-13T17:41:14.831 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 118/121 2024-09-13T17:41:14.832 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 119/121 2024-09-13T17:41:14.832 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 120/121 2024-09-13T17:41:16.988 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 121/121 2024-09-13T17:41:16.989 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:16.989 INFO:teuthology.orchestra.run.smithi102.stdout:Upgraded: 2024-09-13T17:41:16.989 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-13T17:41:16.989 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-13T17:41:16.989 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-13T17:41:16.989 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-13T17:41:16.989 INFO:teuthology.orchestra.run.smithi102.stdout: librados2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:16.989 INFO:teuthology.orchestra.run.smithi102.stdout: librbd1-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:16.989 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-13T17:41:16.989 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-13T17:41:16.989 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-13T17:41:16.989 INFO:teuthology.orchestra.run.smithi102.stdout: bison-3.7.4-5.el9.x86_64 2024-09-13T17:41:16.990 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:16.990 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:16.990 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:16.990 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:16.990 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-grafana-dashboards-2:17.2.7-1627.gbbdadfa3.el9.noarch 2024-09-13T17:41:16.990 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:16.990 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:16.990 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-dashboard-2:17.2.7-1627.gbbdadfa3.el9.noarch 2024-09-13T17:41:16.990 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1627.gbbdadfa3.el9.noarch 2024-09-13T17:41:16.990 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-modules-core-2:17.2.7-1627.gbbdadfa3.el9.noarch 2024-09-13T17:41:16.990 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-rook-2:17.2.7-1627.gbbdadfa3.el9.noarch 2024-09-13T17:41:16.990 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:16.990 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:16.991 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-prometheus-alerts-2:17.2.7-1627.gbbdadfa3.el9.noarch 2024-09-13T17:41:16.991 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:16.991 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-selinux-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:16.991 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:16.991 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-13T17:41:16.991 INFO:teuthology.orchestra.run.smithi102.stdout: flex-2.6.4-9.el9.x86_64 2024-09-13T17:41:16.991 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-13T17:41:16.991 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-13T17:41:16.991 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-13T17:41:16.991 INFO:teuthology.orchestra.run.smithi102.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-13T17:41:16.991 INFO:teuthology.orchestra.run.smithi102.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-13T17:41:16.991 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs-devel-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:16.991 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:16.992 INFO:teuthology.orchestra.run.smithi102.stdout: libcephsqlite-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:16.992 INFO:teuthology.orchestra.run.smithi102.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-13T17:41:16.992 INFO:teuthology.orchestra.run.smithi102.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-13T17:41:16.992 INFO:teuthology.orchestra.run.smithi102.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-13T17:41:16.992 INFO:teuthology.orchestra.run.smithi102.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-13T17:41:16.992 INFO:teuthology.orchestra.run.smithi102.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-13T17:41:16.992 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:16.992 INFO:teuthology.orchestra.run.smithi102.stdout: libradosstriper1-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:16.992 INFO:teuthology.orchestra.run.smithi102.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-13T17:41:16.992 INFO:teuthology.orchestra.run.smithi102.stdout: librgw2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:16.992 INFO:teuthology.orchestra.run.smithi102.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-13T17:41:16.992 INFO:teuthology.orchestra.run.smithi102.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-13T17:41:16.993 INFO:teuthology.orchestra.run.smithi102.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-13T17:41:16.993 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-13T17:41:16.993 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-13T17:41:16.993 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-13T17:41:16.993 INFO:teuthology.orchestra.run.smithi102.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-13T17:41:16.993 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-13T17:41:16.993 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-argparse-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:16.993 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-common-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:16.993 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:16.993 INFO:teuthology.orchestra.run.smithi102.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-13T17:41:16.993 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-13T17:41:16.993 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-13T17:41:16.993 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-13T17:41:16.994 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-13T17:41:16.994 INFO:teuthology.orchestra.run.smithi102.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-13T17:41:16.994 INFO:teuthology.orchestra.run.smithi102.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-13T17:41:16.994 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-13T17:41:16.994 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-13T17:41:16.994 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-13T17:41:16.994 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-13T17:41:16.994 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-13T17:41:16.994 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-13T17:41:16.994 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-13T17:41:16.994 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-13T17:41:16.994 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-13T17:41:16.994 INFO:teuthology.orchestra.run.smithi102.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-13T17:41:16.994 INFO:teuthology.orchestra.run.smithi102.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-13T17:41:16.995 INFO:teuthology.orchestra.run.smithi102.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-13T17:41:16.995 INFO:teuthology.orchestra.run.smithi102.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-13T17:41:16.995 INFO:teuthology.orchestra.run.smithi102.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-13T17:41:16.995 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-13T17:41:16.995 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-13T17:41:16.995 INFO:teuthology.orchestra.run.smithi102.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-13T17:41:16.995 INFO:teuthology.orchestra.run.smithi102.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-13T17:41:16.995 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-13T17:41:16.995 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ply-3.11-14.el9.noarch 2024-09-13T17:41:16.995 INFO:teuthology.orchestra.run.smithi102.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-13T17:41:16.995 INFO:teuthology.orchestra.run.smithi102.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-13T17:41:16.995 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-13T17:41:16.995 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-13T17:41:16.995 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-13T17:41:16.996 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-13T17:41:16.996 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-13T17:41:16.996 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:16.996 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:16.996 INFO:teuthology.orchestra.run.smithi102.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-13T17:41:16.996 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-13T17:41:16.996 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-13T17:41:16.996 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:16.996 INFO:teuthology.orchestra.run.smithi102.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-13T17:41:16.996 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-13T17:41:16.996 INFO:teuthology.orchestra.run.smithi102.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-13T17:41:16.996 INFO:teuthology.orchestra.run.smithi102.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-13T17:41:16.996 INFO:teuthology.orchestra.run.smithi102.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-13T17:41:16.997 INFO:teuthology.orchestra.run.smithi102.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-13T17:41:16.997 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-13T17:41:16.997 INFO:teuthology.orchestra.run.smithi102.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-13T17:41:16.997 INFO:teuthology.orchestra.run.smithi102.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-13T17:41:16.997 INFO:teuthology.orchestra.run.smithi102.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-13T17:41:16.997 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:16.997 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:16.997 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T17:41:16.997 INFO:teuthology.orchestra.run.smithi102.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-13T17:41:16.998 INFO:teuthology.orchestra.run.smithi102.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-13T17:41:16.998 INFO:teuthology.orchestra.run.smithi102.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-13T17:41:16.998 INFO:teuthology.orchestra.run.smithi102.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-13T17:41:16.998 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:16.998 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-13T17:41:17.406 DEBUG:teuthology.parallel:result is None 2024-09-13T17:41:17.406 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T17:41:17.406 INFO:teuthology.packaging:ref: None 2024-09-13T17:41:17.406 INFO:teuthology.packaging:tag: None 2024-09-13T17:41:17.407 INFO:teuthology.packaging:branch: quincy 2024-09-13T17:41:17.407 INFO:teuthology.packaging:sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:41:17.407 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-09-13T17:41:17.641 DEBUG:teuthology.orchestra.run.smithi102:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-13T17:41:17.665 INFO:teuthology.orchestra.run.smithi102.stdout:17.2.7-1627.gbbdadfa3.el9 2024-09-13T17:41:17.665 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1627.gbbdadfa3.el9 2024-09-13T17:41:17.666 INFO:teuthology.task.install:The correct ceph version 17.2.7-1627.gbbdadfa3 is installed. 2024-09-13T17:41:17.667 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T17:41:17.667 INFO:teuthology.packaging:ref: None 2024-09-13T17:41:17.668 INFO:teuthology.packaging:tag: None 2024-09-13T17:41:17.668 INFO:teuthology.packaging:branch: quincy 2024-09-13T17:41:17.668 INFO:teuthology.packaging:sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:41:17.668 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-09-13T17:41:17.873 DEBUG:teuthology.orchestra.run.smithi171:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-13T17:41:17.900 INFO:teuthology.orchestra.run.smithi171.stdout:17.2.7-1627.gbbdadfa3.el9 2024-09-13T17:41:17.900 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1627.gbbdadfa3.el9 2024-09-13T17:41:17.900 INFO:teuthology.task.install:The correct ceph version 17.2.7-1627.gbbdadfa3 is installed. 2024-09-13T17:41:17.902 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-13T17:41:17.902 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-13T17:41:17.902 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T17:41:17.930 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T17:41:17.930 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T17:41:17.971 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-13T17:41:17.971 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-13T17:41:17.971 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T17:41:17.997 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T17:41:18.062 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T17:41:18.063 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T17:41:18.092 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T17:41:18.161 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-13T17:41:18.162 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-13T17:41:18.162 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T17:41:18.187 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T17:41:18.252 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T17:41:18.252 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T17:41:18.281 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T17:41:18.348 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-13T17:41:18.348 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-13T17:41:18.348 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T17:41:18.373 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T17:41:18.438 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T17:41:18.438 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T17:41:18.465 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T17:41:18.532 INFO:teuthology.run_tasks:Running task print... 2024-09-13T17:41:18.541 INFO:teuthology.task.print:**** done installing quincy 2024-09-13T17:41:18.541 INFO:teuthology.run_tasks:Running task ceph... 2024-09-13T17:41:18.621 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-13T17:41:18.621 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 777 /var/log/ceph 2024-09-13T17:41:18.623 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 777 /var/log/ceph 2024-09-13T17:41:18.649 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-13T17:41:18.649 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T17:41:18.688 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T17:41:18.715 INFO:tasks.ceph:Creating extra log directories... 2024-09-13T17:41:18.715 DEBUG:teuthology.orchestra.run.smithi102:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T17:41:18.752 DEBUG:teuthology.orchestra.run.smithi171:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T17:41:18.790 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-13T17:41:18.790 INFO:tasks.ceph:config {'conf': {'global': {'mon warn on pool no app': False, 'ms bind msgr2': False, 'mon pg warn min per osd': 0}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['overall HEALTH_', '\\(FS_', '\\(MDS_', '\\(OSD_', '\\(MON_DOWN\\)', '\\(CACHE_POOL_', '\\(POOL_', '\\(MGR_DOWN\\)', '\\(PG_', '\\(SMALLER_PGP_NUM\\)', 'Monitor daemon marked osd', 'Behind on trimming', 'Manager daemon', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-13T17:41:18.790 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903920', 'branch': 'wip-rishabh-testing-20240910.100730-reef', 'description': 'fs/upgrade/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/quincy 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/new_ops/{newops}}}', 'email': None, 'first_in_suite': False, 'job_id': '7903920', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'meta': [{'desc': 'install ceph/quincy latest\n'}], 'name': 'rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240910.100730-reef'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240910.100730-reef', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3'], ['client.0']], 'seed': 6439, 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'sleep_before_teardown': 0, 'subset': '964/1024', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240910.100730-reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'targets': {'smithi102.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPCxH4XlJVa0qTASLbCPXm4S2jgGhwiQoc+YoAYL+BlUWqlpkFLp/koBmYbttNW3l/qBHSrAaEv41semcbjhuSI=', 'smithi171.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAJmIrHYCHfQDZnEHx3qzeAFlpSvdSb2q7Wo2Y9GFRunr/lWaf62giCTPxT013ljyzUTTqESTllGhoFUI+yCY2I='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'branch': 'quincy', 'exclude_packages': ['cephadm', 'ceph-mgr-cephadm', 'ceph-immutable-object-cache', 'python3-rados', 'python3-rgw', 'python3-rbd', 'python3-cephfs', 'ceph-volume'], 'extra_packages': ['python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'], 'extra_system_packages': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux'], 'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}}, {'print': '**** done installing quincy'}, {'ceph': {'conf': {'global': {'mon warn on pool no app': False, 'ms bind msgr2': False, 'mon pg warn min per osd': 0}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'log-ignorelist': ['overall HEALTH_', '\\(FS_', '\\(MDS_', '\\(OSD_', '\\(MON_DOWN\\)', '\\(CACHE_POOL_', '\\(POOL_', '\\(MGR_DOWN\\)', '\\(PG_', '\\(SMALLER_PGP_NUM\\)', 'Monitor daemon marked osd', 'Behind on trimming', 'Manager daemon', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'flavor': 'default', 'fs': 'xfs', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'cluster': 'ceph'}}, {'exec': {'osd.0': ['ceph osd set-require-min-compat-client quincy']}}, {'print': '**** done ceph'}, {'ceph-fuse': None}, {'ceph-fuse': {'client.0': {'mounted': False}}}, {'print': '**** done unmount client.0'}, {'install.upgrade': {'client.0': None}}, {'print': '**** done install.upgrade on client.0'}, {'ceph.healthy': None}, {'ceph-fuse': {'client.0': None}}, {'print': '**** done remount client.0'}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_newops']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end", 'if not is_fuse() then reject() end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-13_14:42:16', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-13T17:41:18.791 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T17:41:18.817 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T17:41:18.838 DEBUG:teuthology.orchestra.run.smithi102:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T17:41:18.873 DEBUG:teuthology.orchestra.run.smithi171:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T17:41:18.904 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-13T17:41:18.904 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T17:41:18.951 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T17:41:18.951 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_1 2024-09-13T17:41:19.006 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T17:41:19.007 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:41:19.007 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-13T17:41:19.007 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:41:19.007 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T17:41:19.007 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-09-13 17:41:10.102671019 +0000 2024-09-13T17:41:19.007 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-09-13 17:39:27.419779485 +0000 2024-09-13T17:41:19.007 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-09-13 17:39:27.419779485 +0000 2024-09-13T17:41:19.007 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-09-13 17:39:27.419779485 +0000 2024-09-13T17:41:19.007 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T17:41:19.069 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-09-13T17:41:19.070 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-09-13T17:41:19.070 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000208532 s, 2.5 MB/s 2024-09-13T17:41:19.071 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T17:41:19.128 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_2 2024-09-13T17:41:19.183 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T17:41:19.183 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:41:19.183 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-13T17:41:19.183 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:41:19.183 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T17:41:19.183 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-09-13 17:41:10.102671019 +0000 2024-09-13T17:41:19.183 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-09-13 17:39:27.756772604 +0000 2024-09-13T17:41:19.183 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-09-13 17:39:27.756772604 +0000 2024-09-13T17:41:19.183 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-09-13 17:39:27.756772604 +0000 2024-09-13T17:41:19.183 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T17:41:19.245 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-09-13T17:41:19.245 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-09-13T17:41:19.246 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000168862 s, 3.0 MB/s 2024-09-13T17:41:19.247 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T17:41:19.304 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_3 2024-09-13T17:41:19.359 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T17:41:19.359 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:41:19.359 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-13T17:41:19.359 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:41:19.359 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T17:41:19.359 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-09-13 17:41:10.103670999 +0000 2024-09-13T17:41:19.359 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-09-13 17:39:28.093765723 +0000 2024-09-13T17:41:19.359 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-09-13 17:39:28.093765723 +0000 2024-09-13T17:41:19.360 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-09-13 17:39:28.093765723 +0000 2024-09-13T17:41:19.360 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T17:41:19.422 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-09-13T17:41:19.422 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-09-13T17:41:19.422 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000158666 s, 3.2 MB/s 2024-09-13T17:41:19.424 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T17:41:19.479 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_4 2024-09-13T17:41:19.535 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T17:41:19.535 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:41:19.535 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-13T17:41:19.535 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:41:19.535 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T17:41:19.535 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-09-13 17:41:10.103670999 +0000 2024-09-13T17:41:19.535 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-09-13 17:39:28.442758597 +0000 2024-09-13T17:41:19.535 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-09-13 17:39:28.442758597 +0000 2024-09-13T17:41:19.535 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-09-13 17:39:28.442758597 +0000 2024-09-13T17:41:19.536 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T17:41:19.599 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-09-13T17:41:19.599 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-09-13T17:41:19.599 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000185088 s, 2.8 MB/s 2024-09-13T17:41:19.600 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T17:41:19.656 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T17:41:19.656 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi102.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2024-09-13T17:41:19.656 INFO:tasks.ceph:Generating config... 2024-09-13T17:41:19.657 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-13T17:41:19.658 INFO:tasks.ceph:[global] ms bind msgr2 = False 2024-09-13T17:41:19.658 INFO:tasks.ceph:[global] mon pg warn min per osd = 0 2024-09-13T17:41:19.658 INFO:tasks.ceph:[client] client mount timeout = 600 2024-09-13T17:41:19.658 INFO:tasks.ceph:[client] debug client = 20 2024-09-13T17:41:19.658 INFO:tasks.ceph:[client] debug ms = 1 2024-09-13T17:41:19.658 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-09-13T17:41:19.658 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-09-13T17:41:19.658 INFO:tasks.ceph:[mds] debug mds = 20 2024-09-13T17:41:19.658 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-09-13T17:41:19.658 INFO:tasks.ceph:[mds] debug ms = 1 2024-09-13T17:41:19.658 INFO:tasks.ceph:[mds] mds debug frag = True 2024-09-13T17:41:19.658 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-09-13T17:41:19.658 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-09-13T17:41:19.659 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-09-13T17:41:19.659 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-09-13T17:41:19.659 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-09-13T17:41:19.659 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-09-13T17:41:19.659 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-13T17:41:19.659 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-13T17:41:19.659 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-13T17:41:19.659 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-13T17:41:19.659 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-13T17:41:19.659 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-13T17:41:19.659 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-09-13T17:41:19.659 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-09-13T17:41:19.659 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-13T17:41:19.659 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-13T17:41:19.660 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-13T17:41:19.660 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-13T17:41:19.660 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-13T17:41:19.660 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-13T17:41:19.660 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-13T17:41:19.660 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-13T17:41:19.660 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-13T17:41:19.660 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-13T17:41:19.660 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-13T17:41:19.660 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-13T17:41:19.660 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-13T17:41:19.660 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-13T17:41:19.660 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-13T17:41:19.660 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-09-13T17:41:19.661 INFO:tasks.ceph:Setting up mon.a... 2024-09-13T17:41:19.661 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-13T17:41:19.737 INFO:teuthology.orchestra.run.smithi102.stdout:creating /etc/ceph/ceph.keyring 2024-09-13T17:41:19.739 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-13T17:41:19.818 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T17:41:19.883 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.102'), ('mon.b', '[v2:172.21.15.102:3301,v1:172.21.15.102:6790]'), ('mon.c', '[v2:172.21.15.102:3302,v1:172.21.15.102:6791]')] 2024-09-13T17:41:19.883 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.102,[v2:172.21.15.102:3301,v1:172.21.15.102:6790],[v2:172.21.15.102:3302,v1:172.21.15.102:6791]', 'mon warn on pool no app': False, 'ms bind msgr2': False, 'mon pg warn min per osd': 0}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-09-13T17:41:19.884 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-13T17:41:19.884 DEBUG:teuthology.orchestra.run.smithi102:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T17:41:19.938 DEBUG:teuthology.orchestra.run.smithi102:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.102 --addv b '[v2:172.21.15.102:3301,v1:172.21.15.102:6790]' --addv c '[v2:172.21.15.102:3302,v1:172.21.15.102:6791]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:41:20.019 INFO:teuthology.orchestra.run.smithi102.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:41:20.020 INFO:teuthology.orchestra.run.smithi102.stdout:monmaptool: generated fsid 6507a4f0-6c1b-42c0-9c4d-5af24a57491d 2024-09-13T17:41:20.020 INFO:teuthology.orchestra.run.smithi102.stdout:setting min_mon_release = octopus 2024-09-13T17:41:20.020 INFO:teuthology.orchestra.run.smithi102.stdout:epoch 0 2024-09-13T17:41:20.020 INFO:teuthology.orchestra.run.smithi102.stdout:fsid 6507a4f0-6c1b-42c0-9c4d-5af24a57491d 2024-09-13T17:41:20.020 INFO:teuthology.orchestra.run.smithi102.stdout:last_changed 2024-09-13T17:41:20.018918+0000 2024-09-13T17:41:20.020 INFO:teuthology.orchestra.run.smithi102.stdout:created 2024-09-13T17:41:20.018918+0000 2024-09-13T17:41:20.020 INFO:teuthology.orchestra.run.smithi102.stdout:min_mon_release 15 (octopus) 2024-09-13T17:41:20.020 INFO:teuthology.orchestra.run.smithi102.stdout:election_strategy: 1 2024-09-13T17:41:20.020 INFO:teuthology.orchestra.run.smithi102.stdout:0: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.a 2024-09-13T17:41:20.020 INFO:teuthology.orchestra.run.smithi102.stdout:1: [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] mon.b 2024-09-13T17:41:20.020 INFO:teuthology.orchestra.run.smithi102.stdout:2: [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] mon.c 2024-09-13T17:41:20.020 INFO:teuthology.orchestra.run.smithi102.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-13T17:41:20.021 DEBUG:teuthology.orchestra.run.smithi102:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T17:41:20.077 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 6507a4f0-6c1b-42c0-9c4d-5af24a57491d... 2024-09-13T17:41:20.078 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T17:41:20.120 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T17:41:20.160 INFO:teuthology.orchestra.run.smithi102.stdout:[global] 2024-09-13T17:41:20.160 INFO:teuthology.orchestra.run.smithi102.stdout: chdir = "" 2024-09-13T17:41:20.160 INFO:teuthology.orchestra.run.smithi102.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T17:41:20.160 INFO:teuthology.orchestra.run.smithi102.stdout: auth supported = cephx 2024-09-13T17:41:20.160 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:20.160 INFO:teuthology.orchestra.run.smithi102.stdout: filestore xattr use omap = true 2024-09-13T17:41:20.160 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:20.160 INFO:teuthology.orchestra.run.smithi102.stdout: mon clock drift allowed = 1.000 2024-09-13T17:41:20.160 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:20.160 INFO:teuthology.orchestra.run.smithi102.stdout: osd crush chooseleaf type = 0 2024-09-13T17:41:20.160 INFO:teuthology.orchestra.run.smithi102.stdout: auth debug = true 2024-09-13T17:41:20.161 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:20.161 INFO:teuthology.orchestra.run.smithi102.stdout: ms die on old message = true 2024-09-13T17:41:20.161 INFO:teuthology.orchestra.run.smithi102.stdout: ms die on bug = true 2024-09-13T17:41:20.161 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:20.161 INFO:teuthology.orchestra.run.smithi102.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T17:41:20.161 INFO:teuthology.orchestra.run.smithi102.stdout: mon pg warn max object skew = 0 2024-09-13T17:41:20.161 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:20.161 INFO:teuthology.orchestra.run.smithi102.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T17:41:20.161 INFO:teuthology.orchestra.run.smithi102.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T17:41:20.161 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:20.161 INFO:teuthology.orchestra.run.smithi102.stdout: osd pool default size = 2 2024-09-13T17:41:20.161 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:20.161 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd allow primary affinity = true 2024-09-13T17:41:20.161 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd allow pg remap = true 2024-09-13T17:41:20.162 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on legacy crush tunables = false 2024-09-13T17:41:20.162 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on crush straw calc version zero = false 2024-09-13T17:41:20.162 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on no sortbitwise = false 2024-09-13T17:41:20.162 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on osd down out interval zero = false 2024-09-13T17:41:20.162 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on too few osds = false 2024-09-13T17:41:20.162 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T17:41:20.162 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T17:41:20.162 INFO:teuthology.orchestra.run.smithi102.stdout: mon_allow_pool_size_one = true 2024-09-13T17:41:20.162 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:20.162 INFO:teuthology.orchestra.run.smithi102.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T17:41:20.162 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:20.162 INFO:teuthology.orchestra.run.smithi102.stdout: osd default data pool replay window = 5 2024-09-13T17:41:20.162 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:20.162 INFO:teuthology.orchestra.run.smithi102.stdout: mon allow pool delete = true 2024-09-13T17:41:20.163 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:20.163 INFO:teuthology.orchestra.run.smithi102.stdout: mon cluster log file level = debug 2024-09-13T17:41:20.163 INFO:teuthology.orchestra.run.smithi102.stdout: debug asserts on shutdown = true 2024-09-13T17:41:20.163 INFO:teuthology.orchestra.run.smithi102.stdout: mon health detail to clog = false 2024-09-13T17:41:20.163 INFO:teuthology.orchestra.run.smithi102.stdout: mon host = "172.21.15.102,[v2:172.21.15.102:3301,v1:172.21.15.102:6790],[v2:172.21.15.102:3302,v1:172.21.15.102:6791]" 2024-09-13T17:41:20.163 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on pool no app = False 2024-09-13T17:41:20.163 INFO:teuthology.orchestra.run.smithi102.stdout: ms bind msgr2 = False 2024-09-13T17:41:20.163 INFO:teuthology.orchestra.run.smithi102.stdout: mon pg warn min per osd = 0 2024-09-13T17:41:20.163 INFO:teuthology.orchestra.run.smithi102.stdout: fsid = 6507a4f0-6c1b-42c0-9c4d-5af24a57491d 2024-09-13T17:41:20.163 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:20.163 INFO:teuthology.orchestra.run.smithi102.stdout:[osd] 2024-09-13T17:41:20.163 INFO:teuthology.orchestra.run.smithi102.stdout: osd journal size = 100 2024-09-13T17:41:20.163 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:20.163 INFO:teuthology.orchestra.run.smithi102.stdout: osd scrub load threshold = 5.0 2024-09-13T17:41:20.164 INFO:teuthology.orchestra.run.smithi102.stdout: osd scrub max interval = 600 2024-09-13T17:41:20.164 INFO:teuthology.orchestra.run.smithi102.stdout: osd mclock profile = high_recovery_ops 2024-09-13T17:41:20.164 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:20.164 INFO:teuthology.orchestra.run.smithi102.stdout: osd recover clone overlap = true 2024-09-13T17:41:20.164 INFO:teuthology.orchestra.run.smithi102.stdout: osd recovery max chunk = 1048576 2024-09-13T17:41:20.164 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:20.164 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug shutdown = true 2024-09-13T17:41:20.164 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug op order = true 2024-09-13T17:41:20.164 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug verify stray on activate = true 2024-09-13T17:41:20.164 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:20.164 INFO:teuthology.orchestra.run.smithi102.stdout: osd open classes on start = true 2024-09-13T17:41:20.164 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug pg log writeout = true 2024-09-13T17:41:20.164 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:20.164 INFO:teuthology.orchestra.run.smithi102.stdout: osd deep scrub update digest min age = 30 2024-09-13T17:41:20.165 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:20.165 INFO:teuthology.orchestra.run.smithi102.stdout: osd map max advance = 10 2024-09-13T17:41:20.165 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:20.165 INFO:teuthology.orchestra.run.smithi102.stdout: journal zero on create = true 2024-09-13T17:41:20.165 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:20.165 INFO:teuthology.orchestra.run.smithi102.stdout: filestore ondisk finisher threads = 3 2024-09-13T17:41:20.165 INFO:teuthology.orchestra.run.smithi102.stdout: filestore apply finisher threads = 3 2024-09-13T17:41:20.165 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:20.165 INFO:teuthology.orchestra.run.smithi102.stdout: bdev debug aio = true 2024-09-13T17:41:20.165 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug misdirected ops = true 2024-09-13T17:41:20.165 INFO:teuthology.orchestra.run.smithi102.stdout: bdev async discard = True 2024-09-13T17:41:20.165 INFO:teuthology.orchestra.run.smithi102.stdout: bdev enable discard = True 2024-09-13T17:41:20.165 INFO:teuthology.orchestra.run.smithi102.stdout: bluestore allocator = bitmap 2024-09-13T17:41:20.165 INFO:teuthology.orchestra.run.smithi102.stdout: bluestore block size = 96636764160 2024-09-13T17:41:20.165 INFO:teuthology.orchestra.run.smithi102.stdout: bluestore fsck on mount = True 2024-09-13T17:41:20.166 INFO:teuthology.orchestra.run.smithi102.stdout: debug bluefs = 1/20 2024-09-13T17:41:20.166 INFO:teuthology.orchestra.run.smithi102.stdout: debug bluestore = 1/20 2024-09-13T17:41:20.166 INFO:teuthology.orchestra.run.smithi102.stdout: debug ms = 1 2024-09-13T17:41:20.166 INFO:teuthology.orchestra.run.smithi102.stdout: debug osd = 20 2024-09-13T17:41:20.166 INFO:teuthology.orchestra.run.smithi102.stdout: debug rocksdb = 4/10 2024-09-13T17:41:20.166 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T17:41:20.166 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd full ratio = 0.9 2024-09-13T17:41:20.166 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd nearfull ratio = 0.8 2024-09-13T17:41:20.166 INFO:teuthology.orchestra.run.smithi102.stdout: osd failsafe full ratio = 0.95 2024-09-13T17:41:20.166 INFO:teuthology.orchestra.run.smithi102.stdout: osd objectstore = bluestore 2024-09-13T17:41:20.166 INFO:teuthology.orchestra.run.smithi102.stdout: osd op complaint time = 180 2024-09-13T17:41:20.166 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:20.166 INFO:teuthology.orchestra.run.smithi102.stdout:[mgr] 2024-09-13T17:41:20.166 INFO:teuthology.orchestra.run.smithi102.stdout: debug ms = 1 2024-09-13T17:41:20.166 INFO:teuthology.orchestra.run.smithi102.stdout: debug mgr = 20 2024-09-13T17:41:20.167 INFO:teuthology.orchestra.run.smithi102.stdout: debug mon = 20 2024-09-13T17:41:20.167 INFO:teuthology.orchestra.run.smithi102.stdout: debug auth = 20 2024-09-13T17:41:20.167 INFO:teuthology.orchestra.run.smithi102.stdout: mon reweight min pgs per osd = 4 2024-09-13T17:41:20.167 INFO:teuthology.orchestra.run.smithi102.stdout: mon reweight min bytes per osd = 10 2024-09-13T17:41:20.167 INFO:teuthology.orchestra.run.smithi102.stdout: mgr/telemetry/nag = false 2024-09-13T17:41:20.167 INFO:teuthology.orchestra.run.smithi102.stdout: debug client = 20 2024-09-13T17:41:20.167 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:20.167 INFO:teuthology.orchestra.run.smithi102.stdout:[mon] 2024-09-13T17:41:20.167 INFO:teuthology.orchestra.run.smithi102.stdout: debug ms = 1 2024-09-13T17:41:20.167 INFO:teuthology.orchestra.run.smithi102.stdout: debug mon = 20 2024-09-13T17:41:20.167 INFO:teuthology.orchestra.run.smithi102.stdout: debug paxos = 20 2024-09-13T17:41:20.167 INFO:teuthology.orchestra.run.smithi102.stdout: debug auth = 20 2024-09-13T17:41:20.167 INFO:teuthology.orchestra.run.smithi102.stdout: mon data avail warn = 5 2024-09-13T17:41:20.167 INFO:teuthology.orchestra.run.smithi102.stdout: mon mgr mkfs grace = 240 2024-09-13T17:41:20.167 INFO:teuthology.orchestra.run.smithi102.stdout: mon reweight min pgs per osd = 4 2024-09-13T17:41:20.168 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd reporter subtree level = osd 2024-09-13T17:41:20.168 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd prime pg temp = true 2024-09-13T17:41:20.168 INFO:teuthology.orchestra.run.smithi102.stdout: mon reweight min bytes per osd = 10 2024-09-13T17:41:20.168 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:20.168 INFO:teuthology.orchestra.run.smithi102.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T17:41:20.168 INFO:teuthology.orchestra.run.smithi102.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T17:41:20.168 INFO:teuthology.orchestra.run.smithi102.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T17:41:20.168 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:20.168 INFO:teuthology.orchestra.run.smithi102.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T17:41:20.168 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T17:41:20.168 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T17:41:20.168 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:20.168 INFO:teuthology.orchestra.run.smithi102.stdout: # 1m isn't quite enough 2024-09-13T17:41:20.169 INFO:teuthology.orchestra.run.smithi102.stdout: mon_down_mkfs_grace = 2m 2024-09-13T17:41:20.169 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:20.169 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_filestore_osds = false 2024-09-13T17:41:20.169 INFO:teuthology.orchestra.run.smithi102.stdout: mon op complaint time = 120 2024-09-13T17:41:20.169 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on pool no app = False 2024-09-13T17:41:20.169 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:20.169 INFO:teuthology.orchestra.run.smithi102.stdout:[client] 2024-09-13T17:41:20.169 INFO:teuthology.orchestra.run.smithi102.stdout: rgw cache enabled = true 2024-09-13T17:41:20.169 INFO:teuthology.orchestra.run.smithi102.stdout: rgw enable ops log = true 2024-09-13T17:41:20.169 INFO:teuthology.orchestra.run.smithi102.stdout: rgw enable usage log = true 2024-09-13T17:41:20.169 INFO:teuthology.orchestra.run.smithi102.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T17:41:20.169 INFO:teuthology.orchestra.run.smithi102.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T17:41:20.169 INFO:teuthology.orchestra.run.smithi102.stdout: client mount timeout = 600 2024-09-13T17:41:20.169 INFO:teuthology.orchestra.run.smithi102.stdout: debug client = 20 2024-09-13T17:41:20.170 INFO:teuthology.orchestra.run.smithi102.stdout: debug ms = 1 2024-09-13T17:41:20.170 INFO:teuthology.orchestra.run.smithi102.stdout: rados mon op timeout = 900 2024-09-13T17:41:20.170 INFO:teuthology.orchestra.run.smithi102.stdout: rados osd op timeout = 900 2024-09-13T17:41:20.170 INFO:teuthology.orchestra.run.smithi102.stdout:[mon.a] 2024-09-13T17:41:20.170 INFO:teuthology.orchestra.run.smithi102.stdout:[mon.b] 2024-09-13T17:41:20.170 INFO:teuthology.orchestra.run.smithi102.stdout:[mon.c] 2024-09-13T17:41:20.170 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.a] 2024-09-13T17:41:20.170 INFO:teuthology.orchestra.run.smithi102.stdout:[mds] 2024-09-13T17:41:20.170 INFO:teuthology.orchestra.run.smithi102.stdout: debug mds = 20 2024-09-13T17:41:20.170 INFO:teuthology.orchestra.run.smithi102.stdout: debug mds balancer = 20 2024-09-13T17:41:20.170 INFO:teuthology.orchestra.run.smithi102.stdout: debug ms = 1 2024-09-13T17:41:20.170 INFO:teuthology.orchestra.run.smithi102.stdout: mds debug frag = True 2024-09-13T17:41:20.170 INFO:teuthology.orchestra.run.smithi102.stdout: mds debug scatterstat = True 2024-09-13T17:41:20.170 INFO:teuthology.orchestra.run.smithi102.stdout: mds op complaint time = 180 2024-09-13T17:41:20.170 INFO:teuthology.orchestra.run.smithi102.stdout: mds verify scatter = True 2024-09-13T17:41:20.171 INFO:teuthology.orchestra.run.smithi102.stdout: osd op complaint time = 180 2024-09-13T17:41:20.171 INFO:teuthology.orchestra.run.smithi102.stdout: rados mon op timeout = 900 2024-09-13T17:41:20.171 INFO:teuthology.orchestra.run.smithi102.stdout: rados osd op timeout = 900 2024-09-13T17:41:20.171 INFO:teuthology.orchestra.run.smithi171.stdout:[global] 2024-09-13T17:41:20.171 INFO:teuthology.orchestra.run.smithi171.stdout: chdir = "" 2024-09-13T17:41:20.171 INFO:teuthology.orchestra.run.smithi171.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T17:41:20.171 INFO:teuthology.orchestra.run.smithi171.stdout: auth supported = cephx 2024-09-13T17:41:20.171 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:41:20.172 INFO:teuthology.orchestra.run.smithi171.stdout: filestore xattr use omap = true 2024-09-13T17:41:20.172 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:41:20.172 INFO:teuthology.orchestra.run.smithi171.stdout: mon clock drift allowed = 1.000 2024-09-13T17:41:20.172 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:41:20.172 INFO:teuthology.orchestra.run.smithi171.stdout: osd crush chooseleaf type = 0 2024-09-13T17:41:20.172 INFO:teuthology.orchestra.run.smithi171.stdout: auth debug = true 2024-09-13T17:41:20.172 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:41:20.172 INFO:teuthology.orchestra.run.smithi171.stdout: ms die on old message = true 2024-09-13T17:41:20.172 INFO:teuthology.orchestra.run.smithi171.stdout: ms die on bug = true 2024-09-13T17:41:20.172 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:41:20.172 INFO:teuthology.orchestra.run.smithi171.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T17:41:20.172 INFO:teuthology.orchestra.run.smithi171.stdout: mon pg warn max object skew = 0 2024-09-13T17:41:20.172 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:41:20.172 INFO:teuthology.orchestra.run.smithi171.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T17:41:20.173 INFO:teuthology.orchestra.run.smithi171.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T17:41:20.173 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:41:20.173 INFO:teuthology.orchestra.run.smithi171.stdout: osd pool default size = 2 2024-09-13T17:41:20.173 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:41:20.173 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd allow primary affinity = true 2024-09-13T17:41:20.173 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd allow pg remap = true 2024-09-13T17:41:20.173 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on legacy crush tunables = false 2024-09-13T17:41:20.173 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on crush straw calc version zero = false 2024-09-13T17:41:20.173 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on no sortbitwise = false 2024-09-13T17:41:20.173 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on osd down out interval zero = false 2024-09-13T17:41:20.173 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on too few osds = false 2024-09-13T17:41:20.173 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T17:41:20.173 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T17:41:20.174 INFO:teuthology.orchestra.run.smithi171.stdout: mon_allow_pool_size_one = true 2024-09-13T17:41:20.174 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:41:20.174 INFO:teuthology.orchestra.run.smithi171.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T17:41:20.174 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:41:20.174 INFO:teuthology.orchestra.run.smithi171.stdout: osd default data pool replay window = 5 2024-09-13T17:41:20.174 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:41:20.174 INFO:teuthology.orchestra.run.smithi171.stdout: mon allow pool delete = true 2024-09-13T17:41:20.174 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:41:20.174 INFO:teuthology.orchestra.run.smithi171.stdout: mon cluster log file level = debug 2024-09-13T17:41:20.174 INFO:teuthology.orchestra.run.smithi171.stdout: debug asserts on shutdown = true 2024-09-13T17:41:20.174 INFO:teuthology.orchestra.run.smithi171.stdout: mon health detail to clog = false 2024-09-13T17:41:20.174 INFO:teuthology.orchestra.run.smithi171.stdout: mon host = "172.21.15.102,[v2:172.21.15.102:3301,v1:172.21.15.102:6790],[v2:172.21.15.102:3302,v1:172.21.15.102:6791]" 2024-09-13T17:41:20.174 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on pool no app = False 2024-09-13T17:41:20.174 INFO:teuthology.orchestra.run.smithi171.stdout: ms bind msgr2 = False 2024-09-13T17:41:20.174 INFO:teuthology.orchestra.run.smithi171.stdout: mon pg warn min per osd = 0 2024-09-13T17:41:20.175 INFO:teuthology.orchestra.run.smithi171.stdout: fsid = 6507a4f0-6c1b-42c0-9c4d-5af24a57491d 2024-09-13T17:41:20.175 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:41:20.175 INFO:teuthology.orchestra.run.smithi171.stdout:[osd] 2024-09-13T17:41:20.175 INFO:teuthology.orchestra.run.smithi171.stdout: osd journal size = 100 2024-09-13T17:41:20.175 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:41:20.175 INFO:teuthology.orchestra.run.smithi171.stdout: osd scrub load threshold = 5.0 2024-09-13T17:41:20.175 INFO:teuthology.orchestra.run.smithi171.stdout: osd scrub max interval = 600 2024-09-13T17:41:20.175 INFO:teuthology.orchestra.run.smithi171.stdout: osd mclock profile = high_recovery_ops 2024-09-13T17:41:20.175 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:41:20.175 INFO:teuthology.orchestra.run.smithi171.stdout: osd recover clone overlap = true 2024-09-13T17:41:20.175 INFO:teuthology.orchestra.run.smithi171.stdout: osd recovery max chunk = 1048576 2024-09-13T17:41:20.175 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:41:20.175 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug shutdown = true 2024-09-13T17:41:20.176 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug op order = true 2024-09-13T17:41:20.176 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug verify stray on activate = true 2024-09-13T17:41:20.176 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:41:20.176 INFO:teuthology.orchestra.run.smithi171.stdout: osd open classes on start = true 2024-09-13T17:41:20.176 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug pg log writeout = true 2024-09-13T17:41:20.176 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:41:20.176 INFO:teuthology.orchestra.run.smithi171.stdout: osd deep scrub update digest min age = 30 2024-09-13T17:41:20.176 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:41:20.176 INFO:teuthology.orchestra.run.smithi171.stdout: osd map max advance = 10 2024-09-13T17:41:20.176 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:41:20.176 INFO:teuthology.orchestra.run.smithi171.stdout: journal zero on create = true 2024-09-13T17:41:20.176 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:41:20.176 INFO:teuthology.orchestra.run.smithi171.stdout: filestore ondisk finisher threads = 3 2024-09-13T17:41:20.176 INFO:teuthology.orchestra.run.smithi171.stdout: filestore apply finisher threads = 3 2024-09-13T17:41:20.177 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:41:20.177 INFO:teuthology.orchestra.run.smithi171.stdout: bdev debug aio = true 2024-09-13T17:41:20.177 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug misdirected ops = true 2024-09-13T17:41:20.177 INFO:teuthology.orchestra.run.smithi171.stdout: bdev async discard = True 2024-09-13T17:41:20.177 INFO:teuthology.orchestra.run.smithi171.stdout: bdev enable discard = True 2024-09-13T17:41:20.177 INFO:teuthology.orchestra.run.smithi171.stdout: bluestore allocator = bitmap 2024-09-13T17:41:20.177 INFO:teuthology.orchestra.run.smithi171.stdout: bluestore block size = 96636764160 2024-09-13T17:41:20.177 INFO:teuthology.orchestra.run.smithi171.stdout: bluestore fsck on mount = True 2024-09-13T17:41:20.177 INFO:teuthology.orchestra.run.smithi171.stdout: debug bluefs = 1/20 2024-09-13T17:41:20.177 INFO:teuthology.orchestra.run.smithi171.stdout: debug bluestore = 1/20 2024-09-13T17:41:20.177 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-13T17:41:20.177 INFO:teuthology.orchestra.run.smithi171.stdout: debug osd = 20 2024-09-13T17:41:20.177 INFO:teuthology.orchestra.run.smithi171.stdout: debug rocksdb = 4/10 2024-09-13T17:41:20.177 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T17:41:20.178 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd full ratio = 0.9 2024-09-13T17:41:20.178 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd nearfull ratio = 0.8 2024-09-13T17:41:20.178 INFO:teuthology.orchestra.run.smithi171.stdout: osd failsafe full ratio = 0.95 2024-09-13T17:41:20.178 INFO:teuthology.orchestra.run.smithi171.stdout: osd objectstore = bluestore 2024-09-13T17:41:20.178 INFO:teuthology.orchestra.run.smithi171.stdout: osd op complaint time = 180 2024-09-13T17:41:20.178 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:41:20.178 INFO:teuthology.orchestra.run.smithi171.stdout:[mgr] 2024-09-13T17:41:20.178 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-13T17:41:20.178 INFO:teuthology.orchestra.run.smithi171.stdout: debug mgr = 20 2024-09-13T17:41:20.178 INFO:teuthology.orchestra.run.smithi171.stdout: debug mon = 20 2024-09-13T17:41:20.178 INFO:teuthology.orchestra.run.smithi171.stdout: debug auth = 20 2024-09-13T17:41:20.178 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min pgs per osd = 4 2024-09-13T17:41:20.178 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min bytes per osd = 10 2024-09-13T17:41:20.178 INFO:teuthology.orchestra.run.smithi171.stdout: mgr/telemetry/nag = false 2024-09-13T17:41:20.179 INFO:teuthology.orchestra.run.smithi171.stdout: debug client = 20 2024-09-13T17:41:20.179 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:41:20.179 INFO:teuthology.orchestra.run.smithi171.stdout:[mon] 2024-09-13T17:41:20.179 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-13T17:41:20.179 INFO:teuthology.orchestra.run.smithi171.stdout: debug mon = 20 2024-09-13T17:41:20.179 INFO:teuthology.orchestra.run.smithi171.stdout: debug paxos = 20 2024-09-13T17:41:20.179 INFO:teuthology.orchestra.run.smithi171.stdout: debug auth = 20 2024-09-13T17:41:20.179 INFO:teuthology.orchestra.run.smithi171.stdout: mon data avail warn = 5 2024-09-13T17:41:20.179 INFO:teuthology.orchestra.run.smithi171.stdout: mon mgr mkfs grace = 240 2024-09-13T17:41:20.179 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min pgs per osd = 4 2024-09-13T17:41:20.179 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd reporter subtree level = osd 2024-09-13T17:41:20.179 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd prime pg temp = true 2024-09-13T17:41:20.179 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min bytes per osd = 10 2024-09-13T17:41:20.179 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:41:20.180 INFO:teuthology.orchestra.run.smithi171.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T17:41:20.180 INFO:teuthology.orchestra.run.smithi171.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T17:41:20.180 INFO:teuthology.orchestra.run.smithi171.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T17:41:20.180 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:41:20.180 INFO:teuthology.orchestra.run.smithi171.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T17:41:20.180 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T17:41:20.180 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T17:41:20.180 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:41:20.180 INFO:teuthology.orchestra.run.smithi171.stdout: # 1m isn't quite enough 2024-09-13T17:41:20.180 INFO:teuthology.orchestra.run.smithi171.stdout: mon_down_mkfs_grace = 2m 2024-09-13T17:41:20.180 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:41:20.180 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_filestore_osds = false 2024-09-13T17:41:20.180 INFO:teuthology.orchestra.run.smithi171.stdout: mon op complaint time = 120 2024-09-13T17:41:20.180 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on pool no app = False 2024-09-13T17:41:20.180 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:41:20.181 INFO:teuthology.orchestra.run.smithi171.stdout:[client] 2024-09-13T17:41:20.181 INFO:teuthology.orchestra.run.smithi171.stdout: rgw cache enabled = true 2024-09-13T17:41:20.181 INFO:teuthology.orchestra.run.smithi171.stdout: rgw enable ops log = true 2024-09-13T17:41:20.181 INFO:teuthology.orchestra.run.smithi171.stdout: rgw enable usage log = true 2024-09-13T17:41:20.181 INFO:teuthology.orchestra.run.smithi171.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T17:41:20.181 INFO:teuthology.orchestra.run.smithi171.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T17:41:20.181 INFO:teuthology.orchestra.run.smithi171.stdout: client mount timeout = 600 2024-09-13T17:41:20.181 INFO:teuthology.orchestra.run.smithi171.stdout: debug client = 20 2024-09-13T17:41:20.181 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-13T17:41:20.181 INFO:teuthology.orchestra.run.smithi171.stdout: rados mon op timeout = 900 2024-09-13T17:41:20.181 INFO:teuthology.orchestra.run.smithi171.stdout: rados osd op timeout = 900 2024-09-13T17:41:20.181 INFO:teuthology.orchestra.run.smithi171.stdout:[mon.a] 2024-09-13T17:41:20.181 INFO:teuthology.orchestra.run.smithi171.stdout:[mon.b] 2024-09-13T17:41:20.182 INFO:teuthology.orchestra.run.smithi171.stdout:[mon.c] 2024-09-13T17:41:20.182 INFO:teuthology.orchestra.run.smithi171.stdout:[mds.a] 2024-09-13T17:41:20.182 INFO:teuthology.orchestra.run.smithi171.stdout:[mds] 2024-09-13T17:41:20.182 INFO:teuthology.orchestra.run.smithi171.stdout: debug mds = 20 2024-09-13T17:41:20.182 INFO:teuthology.orchestra.run.smithi171.stdout: debug mds balancer = 20 2024-09-13T17:41:20.182 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-13T17:41:20.182 INFO:teuthology.orchestra.run.smithi171.stdout: mds debug frag = True 2024-09-13T17:41:20.182 INFO:teuthology.orchestra.run.smithi171.stdout: mds debug scatterstat = True 2024-09-13T17:41:20.182 INFO:teuthology.orchestra.run.smithi171.stdout: mds op complaint time = 180 2024-09-13T17:41:20.182 INFO:teuthology.orchestra.run.smithi171.stdout: mds verify scatter = True 2024-09-13T17:41:20.182 INFO:teuthology.orchestra.run.smithi171.stdout: osd op complaint time = 180 2024-09-13T17:41:20.182 INFO:teuthology.orchestra.run.smithi171.stdout: rados mon op timeout = 900 2024-09-13T17:41:20.182 INFO:teuthology.orchestra.run.smithi171.stdout: rados osd op timeout = 900 2024-09-13T17:41:20.184 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-13T17:41:20.184 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-13T17:41:20.273 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-13T17:41:20.273 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-13T17:41:20.273 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-13T17:41:20.288 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-13T17:41:20.288 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-13T17:41:20.344 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi102.front.sepia.ceph.com 2024-09-13T17:41:20.344 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-13T17:41:20.344 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T17:41:20.344 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T17:41:20.418 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-13T17:41:20.418 DEBUG:teuthology.orchestra.run.smithi102:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:41:20.474 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi171.front.sepia.ceph.com 2024-09-13T17:41:20.474 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T17:41:20.474 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T17:41:20.474 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T17:41:20.513 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T17:41:20.513 DEBUG:teuthology.orchestra.run.smithi171:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:41:20.569 INFO:tasks.ceph:Setting up mon nodes... 2024-09-13T17:41:20.569 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-13T17:41:20.569 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T17:41:20.616 INFO:teuthology.orchestra.run.smithi102.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T17:41:20.619 INFO:tasks.ceph:Setting up mds nodes... 2024-09-13T17:41:20.619 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-13T17:41:20.666 INFO:teuthology.orchestra.run.smithi102.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-13T17:41:20.668 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-13T17:41:20.693 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-13T17:41:20.693 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-13T17:41:20.738 INFO:teuthology.orchestra.run.smithi171.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-13T17:41:20.750 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-13T17:41:20.750 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi102.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2024-09-13T17:41:20.751 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-13T17:41:20.775 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T17:41:20.775 INFO:tasks.ceph:role: osd.0 2024-09-13T17:41:20.775 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi102.front.sepia.ceph.com 2024-09-13T17:41:20.775 DEBUG:teuthology.orchestra.run.smithi102:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T17:41:20.847 INFO:teuthology.orchestra.run.smithi102.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T17:41:20.847 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T17:41:20.847 INFO:teuthology.orchestra.run.smithi102.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T17:41:20.848 INFO:teuthology.orchestra.run.smithi102.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T17:41:20.848 INFO:teuthology.orchestra.run.smithi102.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T17:41:20.848 INFO:teuthology.orchestra.run.smithi102.stdout: = sunit=0 swidth=0 blks 2024-09-13T17:41:20.848 INFO:teuthology.orchestra.run.smithi102.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T17:41:20.848 INFO:teuthology.orchestra.run.smithi102.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T17:41:20.848 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T17:41:20.848 INFO:teuthology.orchestra.run.smithi102.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T17:41:21.348 INFO:teuthology.orchestra.run.smithi102.stdout:Discarding blocks...Done. 2024-09-13T17:41:21.350 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi102.front.sepia.ceph.com -o noatime 2024-09-13T17:41:21.350 DEBUG:teuthology.orchestra.run.smithi102:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-13T17:41:21.492 DEBUG:teuthology.orchestra.run.smithi102:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-13T17:41:21.523 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-13T17:41:21.588 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T17:41:21.589 INFO:tasks.ceph:role: osd.1 2024-09-13T17:41:21.589 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi102.front.sepia.ceph.com 2024-09-13T17:41:21.589 DEBUG:teuthology.orchestra.run.smithi102:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T17:41:21.660 INFO:teuthology.orchestra.run.smithi102.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T17:41:21.660 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T17:41:21.660 INFO:teuthology.orchestra.run.smithi102.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T17:41:21.660 INFO:teuthology.orchestra.run.smithi102.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T17:41:21.661 INFO:teuthology.orchestra.run.smithi102.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T17:41:21.661 INFO:teuthology.orchestra.run.smithi102.stdout: = sunit=0 swidth=0 blks 2024-09-13T17:41:21.661 INFO:teuthology.orchestra.run.smithi102.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T17:41:21.661 INFO:teuthology.orchestra.run.smithi102.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T17:41:21.661 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T17:41:21.661 INFO:teuthology.orchestra.run.smithi102.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T17:41:22.115 INFO:teuthology.orchestra.run.smithi102.stdout:Discarding blocks...Done. 2024-09-13T17:41:22.116 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi102.front.sepia.ceph.com -o noatime 2024-09-13T17:41:22.117 DEBUG:teuthology.orchestra.run.smithi102:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-13T17:41:22.204 DEBUG:teuthology.orchestra.run.smithi102:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-13T17:41:22.233 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-13T17:41:22.298 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T17:41:22.298 INFO:tasks.ceph:role: osd.2 2024-09-13T17:41:22.298 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi102.front.sepia.ceph.com 2024-09-13T17:41:22.298 DEBUG:teuthology.orchestra.run.smithi102:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T17:41:22.370 INFO:teuthology.orchestra.run.smithi102.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T17:41:22.371 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T17:41:22.371 INFO:teuthology.orchestra.run.smithi102.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T17:41:22.371 INFO:teuthology.orchestra.run.smithi102.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T17:41:22.371 INFO:teuthology.orchestra.run.smithi102.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T17:41:22.371 INFO:teuthology.orchestra.run.smithi102.stdout: = sunit=0 swidth=0 blks 2024-09-13T17:41:22.371 INFO:teuthology.orchestra.run.smithi102.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T17:41:22.371 INFO:teuthology.orchestra.run.smithi102.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T17:41:22.371 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T17:41:22.371 INFO:teuthology.orchestra.run.smithi102.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T17:41:22.859 INFO:teuthology.orchestra.run.smithi102.stdout:Discarding blocks...Done. 2024-09-13T17:41:22.861 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi102.front.sepia.ceph.com -o noatime 2024-09-13T17:41:22.861 DEBUG:teuthology.orchestra.run.smithi102:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-13T17:41:23.013 DEBUG:teuthology.orchestra.run.smithi102:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-13T17:41:23.045 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-13T17:41:23.112 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T17:41:23.113 INFO:tasks.ceph:role: osd.3 2024-09-13T17:41:23.113 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi102.front.sepia.ceph.com 2024-09-13T17:41:23.113 DEBUG:teuthology.orchestra.run.smithi102:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T17:41:23.185 INFO:teuthology.orchestra.run.smithi102.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T17:41:23.185 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T17:41:23.186 INFO:teuthology.orchestra.run.smithi102.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T17:41:23.186 INFO:teuthology.orchestra.run.smithi102.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T17:41:23.186 INFO:teuthology.orchestra.run.smithi102.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T17:41:23.186 INFO:teuthology.orchestra.run.smithi102.stdout: = sunit=0 swidth=0 blks 2024-09-13T17:41:23.186 INFO:teuthology.orchestra.run.smithi102.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T17:41:23.186 INFO:teuthology.orchestra.run.smithi102.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T17:41:23.186 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T17:41:23.186 INFO:teuthology.orchestra.run.smithi102.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T17:41:23.719 INFO:teuthology.orchestra.run.smithi102.stdout:Discarding blocks...Done. 2024-09-13T17:41:23.722 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi102.front.sepia.ceph.com -o noatime 2024-09-13T17:41:23.723 DEBUG:teuthology.orchestra.run.smithi102:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-13T17:41:23.879 DEBUG:teuthology.orchestra.run.smithi102:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-13T17:41:23.914 DEBUG:teuthology.orchestra.run.smithi102:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:41:24.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:24.023+0000 7fbcb4a35440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-13T17:41:24.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:24.024+0000 7fbcb4a35440 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-13T17:41:24.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:24.024+0000 7fbcb4a35440 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T17:41:24.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:24.025+0000 7fbcb4a35440 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T17:41:24.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:24.025+0000 7fbcb4a35440 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T17:41:24.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:24.025+0000 7fbcb4a35440 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-13T17:41:24.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:24.551+0000 7fbcb4a35440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T17:41:25.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:25.315+0000 7fbcb4a35440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T17:41:26.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:26.077+0000 7fbcb4a35440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T17:41:26.880 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-13T17:41:26.909 DEBUG:teuthology.orchestra.run.smithi102:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:41:26.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:26.993+0000 7f58ef00c440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-13T17:41:26.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:26.993+0000 7f58ef00c440 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-13T17:41:26.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:26.993+0000 7f58ef00c440 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T17:41:26.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:26.993+0000 7f58ef00c440 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T17:41:26.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:26.993+0000 7f58ef00c440 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T17:41:26.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:26.994+0000 7f58ef00c440 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-13T17:41:27.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:27.518+0000 7f58ef00c440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T17:41:28.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:28.283+0000 7f58ef00c440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T17:41:29.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:29.066+0000 7f58ef00c440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T17:41:29.864 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-13T17:41:29.892 DEBUG:teuthology.orchestra.run.smithi102:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:41:29.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:29.972+0000 7f55e2832440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-13T17:41:29.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:29.972+0000 7f55e2832440 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-13T17:41:29.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:29.972+0000 7f55e2832440 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T17:41:29.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:29.972+0000 7f55e2832440 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T17:41:29.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:29.972+0000 7f55e2832440 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T17:41:29.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:29.973+0000 7f55e2832440 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-13T17:41:30.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:30.498+0000 7f55e2832440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T17:41:31.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:31.263+0000 7f55e2832440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T17:41:32.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:32.025+0000 7f55e2832440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T17:41:32.823 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-13T17:41:32.893 DEBUG:teuthology.orchestra.run.smithi102:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:41:32.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:32.977+0000 7f40dd08b440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-13T17:41:32.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:32.977+0000 7f40dd08b440 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-13T17:41:32.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:32.977+0000 7f40dd08b440 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T17:41:32.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:32.977+0000 7f40dd08b440 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T17:41:32.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:32.977+0000 7f40dd08b440 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T17:41:32.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:32.978+0000 7f40dd08b440 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-13T17:41:33.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:33.502+0000 7f40dd08b440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T17:41:34.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:34.270+0000 7f40dd08b440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T17:41:35.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:35.036+0000 7f40dd08b440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T17:41:35.833 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-13T17:41:35.863 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-13T17:41:35.863 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-13T17:41:35.864 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-13T17:41:35.932 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-13T17:41:35.932 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-13T17:41:35.998 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-13T17:41:35.998 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-13T17:41:36.065 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-13T17:41:36.065 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-13T17:41:36.130 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-13T17:41:36.130 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-13T17:41:36.195 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-13T17:41:36.196 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-13T17:41:36.260 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T17:41:36.261 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-13T17:41:36.281 INFO:tasks.ceph:Adding keys to all mons... 2024-09-13T17:41:36.281 DEBUG:teuthology.orchestra.run.smithi102:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T17:41:36.323 INFO:teuthology.orchestra.run.smithi102.stdout:[mgr.x] 2024-09-13T17:41:36.324 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQBAeeRm9Zm5JBAA1j3HzJzDG8zz5XNmJ9kTAg== 2024-09-13T17:41:36.324 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.a] 2024-09-13T17:41:36.324 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQBAeeRm+mivJxAAV6KWk8hYJYWOfdjhXMjw+A== 2024-09-13T17:41:36.324 INFO:teuthology.orchestra.run.smithi102.stdout:[osd.0] 2024-09-13T17:41:36.324 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQBEeeRmE/BzARAAilYkvotOaS6Fo5QC2J33yQ== 2024-09-13T17:41:36.324 INFO:teuthology.orchestra.run.smithi102.stdout:[osd.1] 2024-09-13T17:41:36.324 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQBGeeRm3hZIOxAAE2ewM4Z40LxLSuqvF/+aow== 2024-09-13T17:41:36.324 INFO:teuthology.orchestra.run.smithi102.stdout:[osd.2] 2024-09-13T17:41:36.324 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQBJeeRmc+cIOhAAbSGkIDTFg7HyaVJTs0alxA== 2024-09-13T17:41:36.324 INFO:teuthology.orchestra.run.smithi102.stdout:[osd.3] 2024-09-13T17:41:36.324 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQBMeeRmahZPOhAA9ni0BvesPpGsHJAof7+Psg== 2024-09-13T17:41:36.324 INFO:teuthology.orchestra.run.smithi102.stdout:[client.0] 2024-09-13T17:41:36.324 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQBAeeRmt/H1KxAAHEY0kvAg5ouvDN3LtMj2KA== 2024-09-13T17:41:36.325 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T17:41:36.411 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T17:41:36.460 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:41:36.506 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:41:36.551 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:41:36.597 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:41:36.644 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T17:41:36.692 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-13T17:41:36.693 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-13T17:41:36.717 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T17:41:36.808 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-13T17:41:36.833 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-13T17:41:36.897 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T17:41:36.988 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-13T17:41:37.013 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-13T17:41:37.078 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T17:41:37.169 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-13T17:41:37.194 DEBUG:teuthology.orchestra.run.smithi102:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:41:37.249 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-13T17:41:37.250 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T17:41:37.250 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T17:41:37.292 INFO:tasks.ceph.mon.a:Started 2024-09-13T17:41:37.292 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-13T17:41:37.292 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-13T17:41:37.296 INFO:tasks.ceph.mon.b:Started 2024-09-13T17:41:37.296 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-13T17:41:37.296 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-13T17:41:37.298 INFO:tasks.ceph.mon.c:Started 2024-09-13T17:41:37.298 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-13T17:41:37.298 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-13T17:41:37.299 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-13T17:41:37.301 INFO:tasks.ceph.mgr.x:Started 2024-09-13T17:41:37.301 INFO:tasks.ceph:Setting crush tunables to default 2024-09-13T17:41:37.301 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd crush tunables default 2024-09-13T17:41:37.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:37.422+0000 7f7822480640 1 Processor -- start 2024-09-13T17:41:37.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:37.422+0000 7f7822480640 1 -- start start 2024-09-13T17:41:37.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:37.422+0000 7f7822480640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f781c09ffd0 0x7f781c12c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:37.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:37.422+0000 7f7822480640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f781c12caf0 0x7f781c09d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:37.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:37.423+0000 7f7822480640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f781c098820 con 0x7f781c09dc00 2024-09-13T17:41:37.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:37.423+0000 7f7822480640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f781c0a03a0 con 0x7f781c09ffd0 2024-09-13T17:41:37.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:37.423+0000 7f7822480640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f781c097900 con 0x7f781c12caf0 2024-09-13T17:41:37.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:37.423+0000 7f781bfff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f781c09ffd0 0x7f781c12c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:37.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:37.423+0000 7f7820a1d640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f781c09dc00 0x7f781c133540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:46006/0 (socket says 172.21.15.102:46006) 2024-09-13T17:41:37.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:37.423+0000 7f7820a1d640 1 -- 172.21.15.102:0/3490410675 learned_addr learned my addr 172.21.15.102:0/3490410675 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:41:37.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:37.423+0000 7f7813fff640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f781c12caf0 0x7f781c09d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:37.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:37.423+0000 7f781bfff640 1 -- 172.21.15.102:0/3490410675 >> v2:172.21.15.102:3300/0 conn(0x7f781c09ffd0 msgr2=0x7f781c12c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:41:37.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:37.423+0000 7f781bfff640 1 -- 172.21.15.102:0/3490410675 >> v2:172.21.15.102:3300/0 conn(0x7f781c09ffd0 msgr2=0x7f781c12c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:41:37.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:37.423+0000 7f781bfff640 1 --2- 172.21.15.102:0/3490410675 >> v2:172.21.15.102:3300/0 conn(0x7f781c09ffd0 0x7f781c12c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T17:41:37.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:37.423+0000 7f781bfff640 1 --2- 172.21.15.102:0/3490410675 >> v2:172.21.15.102:3300/0 conn(0x7f781c09ffd0 0x7f781c12c5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:41:37.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:37.423+0000 7f7813fff640 1 -- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f781c12caf0 msgr2=0x7f781c09d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:41:37.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:37.423+0000 7f7813fff640 1 -- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f781c12caf0 msgr2=0x7f781c09d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:41:37.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:37.423+0000 7f7813fff640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f781c12caf0 0x7f781c09d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T17:41:37.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:37.423+0000 7f7813fff640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f781c12caf0 0x7f781c09d6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:41:37.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:37.624+0000 7f781bfff640 1 --2- 172.21.15.102:0/3490410675 >> v2:172.21.15.102:3300/0 conn(0x7f781c09ffd0 0x7f781c12c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:37.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:37.624+0000 7f7813fff640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f781c12caf0 0x7f781c09d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:37.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:37.624+0000 7f781bfff640 1 -- 172.21.15.102:0/3490410675 >> v2:172.21.15.102:3300/0 conn(0x7f781c09ffd0 msgr2=0x7f781c12c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:41:37.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:37.624+0000 7f781bfff640 1 -- 172.21.15.102:0/3490410675 >> v2:172.21.15.102:3300/0 conn(0x7f781c09ffd0 msgr2=0x7f781c12c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:41:37.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:37.624+0000 7f781bfff640 1 --2- 172.21.15.102:0/3490410675 >> v2:172.21.15.102:3300/0 conn(0x7f781c09ffd0 0x7f781c12c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T17:41:37.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:37.624+0000 7f7813fff640 1 -- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f781c12caf0 msgr2=0x7f781c09d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:41:37.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:37.624+0000 7f7813fff640 1 -- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f781c12caf0 msgr2=0x7f781c09d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:41:37.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:37.624+0000 7f7813fff640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f781c12caf0 0x7f781c09d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T17:41:37.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:37.624+0000 7f781bfff640 1 --2- 172.21.15.102:0/3490410675 >> v2:172.21.15.102:3300/0 conn(0x7f781c09ffd0 0x7f781c12c5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:41:37.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:37.624+0000 7f7813fff640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f781c12caf0 0x7f781c09d6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:41:38.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:38.025+0000 7f781bfff640 1 --2- 172.21.15.102:0/3490410675 >> v2:172.21.15.102:3300/0 conn(0x7f781c09ffd0 0x7f781c12c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:38.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:38.025+0000 7f7813fff640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f781c12caf0 0x7f781c09d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:38.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:38.025+0000 7f781bfff640 1 -- 172.21.15.102:0/3490410675 >> v2:172.21.15.102:3300/0 conn(0x7f781c09ffd0 msgr2=0x7f781c12c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:41:38.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:38.025+0000 7f781bfff640 1 -- 172.21.15.102:0/3490410675 >> v2:172.21.15.102:3300/0 conn(0x7f781c09ffd0 msgr2=0x7f781c12c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:41:38.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:38.025+0000 7f781bfff640 1 --2- 172.21.15.102:0/3490410675 >> v2:172.21.15.102:3300/0 conn(0x7f781c09ffd0 0x7f781c12c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T17:41:38.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:38.025+0000 7f7813fff640 1 -- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f781c12caf0 msgr2=0x7f781c09d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:41:38.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:38.025+0000 7f7813fff640 1 -- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f781c12caf0 msgr2=0x7f781c09d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:41:38.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:38.025+0000 7f7813fff640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f781c12caf0 0x7f781c09d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T17:41:38.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:38.025+0000 7f781bfff640 1 --2- 172.21.15.102:0/3490410675 >> v2:172.21.15.102:3300/0 conn(0x7f781c09ffd0 0x7f781c12c5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:41:38.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:38.025+0000 7f7813fff640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f781c12caf0 0x7f781c09d6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:41:38.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:38.826+0000 7f781bfff640 1 --2- 172.21.15.102:0/3490410675 >> v2:172.21.15.102:3300/0 conn(0x7f781c09ffd0 0x7f781c12c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:38.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:38.826+0000 7f7813fff640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f781c12caf0 0x7f781c09d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:38.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:38.826+0000 7f781bfff640 1 -- 172.21.15.102:0/3490410675 >> v2:172.21.15.102:3300/0 conn(0x7f781c09ffd0 msgr2=0x7f781c12c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:41:38.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:38.826+0000 7f781bfff640 1 -- 172.21.15.102:0/3490410675 >> v2:172.21.15.102:3300/0 conn(0x7f781c09ffd0 msgr2=0x7f781c12c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:41:38.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:38.826+0000 7f781bfff640 1 --2- 172.21.15.102:0/3490410675 >> v2:172.21.15.102:3300/0 conn(0x7f781c09ffd0 0x7f781c12c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T17:41:38.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:38.826+0000 7f7813fff640 1 -- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f781c12caf0 msgr2=0x7f781c09d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:41:38.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:38.826+0000 7f7813fff640 1 -- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f781c12caf0 msgr2=0x7f781c09d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:41:38.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:38.826+0000 7f7813fff640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f781c12caf0 0x7f781c09d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T17:41:38.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:38.826+0000 7f781bfff640 1 --2- 172.21.15.102:0/3490410675 >> v2:172.21.15.102:3300/0 conn(0x7f781c09ffd0 0x7f781c12c5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T17:41:38.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:38.826+0000 7f7813fff640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f781c12caf0 0x7f781c09d6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T17:41:40.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:40.423+0000 7f781a7fc640 1 -- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f781c12caf0 msgr2=0x7f781c09d6c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:41:40.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:40.423+0000 7f781a7fc640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f781c12caf0 0x7f781c09d6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:40.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:40.423+0000 7f781a7fc640 1 -- 172.21.15.102:0/3490410675 >> v2:172.21.15.102:3300/0 conn(0x7f781c09ffd0 msgr2=0x7f781c12c5b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:41:40.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:40.423+0000 7f781a7fc640 1 --2- 172.21.15.102:0/3490410675 >> v2:172.21.15.102:3300/0 conn(0x7f781c09ffd0 0x7f781c12c5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:40.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:40.423+0000 7f781a7fc640 1 -- 172.21.15.102:0/3490410675 >> v1:172.21.15.102:6789/0 conn(0x7f781c09dc00 legacy=0x7f781c133540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:40.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:40.423+0000 7f781a7fc640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77fc003630 0x7f77fc005ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:40.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:40.423+0000 7f781a7fc640 1 -- 172.21.15.102:0/3490410675 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f77fc0060c0 con 0x7f77fc003630 2024-09-13T17:41:40.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:40.423+0000 7f781bfff640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77fc003630 0x7f77fc005ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:40.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:40.423+0000 7f781bfff640 1 -- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77fc003630 msgr2=0x7f77fc005ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:41:40.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:40.423+0000 7f781bfff640 1 -- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77fc003630 msgr2=0x7f77fc005ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:41:40.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:40.423+0000 7f781bfff640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77fc003630 0x7f77fc005ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T17:41:40.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:40.423+0000 7f781bfff640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77fc003630 0x7f77fc005ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:41:40.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:40.624+0000 7f781bfff640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77fc003630 0x7f77fc005ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:40.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:40.624+0000 7f781bfff640 1 -- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77fc003630 msgr2=0x7f77fc005ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:41:40.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:40.624+0000 7f781bfff640 1 -- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77fc003630 msgr2=0x7f77fc005ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:41:40.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:40.624+0000 7f781bfff640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77fc003630 0x7f77fc005ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T17:41:40.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:40.624+0000 7f781bfff640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77fc003630 0x7f77fc005ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:41:41.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:41.025+0000 7f781bfff640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77fc003630 0x7f77fc005ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:41.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:41.025+0000 7f781bfff640 1 -- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77fc003630 msgr2=0x7f77fc005ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:41:41.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:41.025+0000 7f781bfff640 1 -- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77fc003630 msgr2=0x7f77fc005ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:41:41.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:41.025+0000 7f781bfff640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77fc003630 0x7f77fc005ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T17:41:41.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:41.025+0000 7f781bfff640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77fc003630 0x7f77fc005ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:41:41.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:41.826+0000 7f781bfff640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77fc003630 0x7f77fc005ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:41.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:41.826+0000 7f781bfff640 1 -- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77fc003630 msgr2=0x7f77fc005ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:41:41.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:41.826+0000 7f781bfff640 1 -- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77fc003630 msgr2=0x7f77fc005ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:41:41.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:41.826+0000 7f781bfff640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77fc003630 0x7f77fc005ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T17:41:41.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:41.826+0000 7f781bfff640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77fc003630 0x7f77fc005ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T17:41:43.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.422+0000 7f781a7fc640 1 -- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77fc003630 msgr2=0x7f77fc005ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:41:43.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.422+0000 7f781a7fc640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77fc003630 0x7f77fc005ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:43.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.422+0000 7f781a7fc640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f781c12caf0 0x7f77fc0083f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:43.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.422+0000 7f781a7fc640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77fc003630 0x7f77fc00c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:43.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.422+0000 7f781a7fc640 1 -- 172.21.15.102:0/3490410675 --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f781c096e80 con 0x7f781c09dc00 2024-09-13T17:41:43.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.422+0000 7f781a7fc640 1 -- 172.21.15.102:0/3490410675 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77fc0060c0 con 0x7f781c12caf0 2024-09-13T17:41:43.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.422+0000 7f781a7fc640 1 -- 172.21.15.102:0/3490410675 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f77fc005b80 con 0x7f77fc003630 2024-09-13T17:41:43.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.422+0000 7f781bfff640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f781c12caf0 0x7f77fc0083f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:43.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.422+0000 7f7820a1d640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77fc003630 0x7f77fc00c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:43.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.422+0000 7f781b7fe640 1 -- 172.21.15.102:0/3490410675 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 909071834 0 0) 0x7f781c096e80 con 0x7f781c09dc00 2024-09-13T17:41:43.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.422+0000 7f7820a1d640 1 -- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f781c12caf0 msgr2=0x7f77fc0083f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:41:43.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.422+0000 7f7820a1d640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f781c12caf0 0x7f77fc0083f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:43.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.422+0000 7f7820a1d640 1 -- 172.21.15.102:0/3490410675 >> v1:172.21.15.102:6789/0 conn(0x7f781c09dc00 legacy=0x7f77fc00a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:43.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.423+0000 7f7820a1d640 1 -- 172.21.15.102:0/3490410675 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77fc005d10 con 0x7f77fc003630 2024-09-13T17:41:43.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.423+0000 7f7820a1d640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77fc003630 0x7f77fc00c0f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7814002100 tx=0x7f7814001bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80207c4728fe907e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:43.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.424+0000 7f781b7fe640 1 -- 172.21.15.102:0/3490410675 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78140070f0 con 0x7f77fc003630 2024-09-13T17:41:43.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.424+0000 7f781b7fe640 1 -- 172.21.15.102:0/3490410675 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7814007250 con 0x7f77fc003630 2024-09-13T17:41:43.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.424+0000 7f781b7fe640 1 -- 172.21.15.102:0/3490410675 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78140075f0 con 0x7f77fc003630 2024-09-13T17:41:43.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.424+0000 7f7822480640 1 -- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77fc003630 msgr2=0x7f77fc00c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:43.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.424+0000 7f7822480640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77fc003630 0x7f77fc00c0f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7814002100 tx=0x7f7814001bb0 comp rx=0 tx=0).stop 2024-09-13T17:41:43.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.424+0000 7f7822480640 1 -- 172.21.15.102:0/3490410675 shutdown_connections 2024-09-13T17:41:43.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.424+0000 7f7822480640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77fc003630 0x7f77fc00c0f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:43.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.424+0000 7f7822480640 1 --2- 172.21.15.102:0/3490410675 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f781c12caf0 0x7f77fc0083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:43.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.424+0000 7f7822480640 1 --2- 172.21.15.102:0/3490410675 >> v2:172.21.15.102:3300/0 conn(0x7f781c09ffd0 0x7f781c12c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:43.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.424+0000 7f7822480640 1 -- 172.21.15.102:0/3490410675 >> 172.21.15.102:0/3490410675 conn(0x7f781c05e200 msgr2=0x7f781c09f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:43.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.425+0000 7f7822480640 1 -- 172.21.15.102:0/3490410675 shutdown_connections 2024-09-13T17:41:43.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.425+0000 7f7822480640 1 -- 172.21.15.102:0/3490410675 wait complete. 2024-09-13T17:41:43.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.425+0000 7f7822480640 1 Processor -- start 2024-09-13T17:41:43.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.426+0000 7f7822480640 1 -- start start 2024-09-13T17:41:43.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.426+0000 7f7822480640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77fc003630 0x7f781c136c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:43.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.426+0000 7f7822480640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f781c09dc00 0x7f781c130020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:43.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.426+0000 7f7822480640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f781c09ffd0 0x7f781c130560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:43.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.426+0000 7f7822480640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781c133a40 con 0x7f77fc003630 2024-09-13T17:41:43.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.426+0000 7f7822480640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f781c137670 con 0x7f781c09dc00 2024-09-13T17:41:43.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.426+0000 7f7822480640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f781c097050 con 0x7f781c09ffd0 2024-09-13T17:41:43.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.426+0000 7f781bfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77fc003630 0x7f781c136c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:43.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.426+0000 7f7813fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f781c09dc00 0x7f781c130020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:43.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.426+0000 7f781bfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77fc003630 0x7f781c136c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34184/0 (socket says 172.21.15.102:34184) 2024-09-13T17:41:43.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.426+0000 7f781bfff640 1 -- 172.21.15.102:0/2405837070 learned_addr learned my addr 172.21.15.102:0/2405837070 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:41:43.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.426+0000 7f7820a1d640 1 --2- 172.21.15.102:0/2405837070 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f781c09ffd0 0x7f781c130560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:43.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.427+0000 7f7813fff640 1 -- 172.21.15.102:0/2405837070 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f781c09ffd0 msgr2=0x7f781c130560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:43.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.427+0000 7f7813fff640 1 --2- 172.21.15.102:0/2405837070 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f781c09ffd0 0x7f781c130560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:43.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.427+0000 7f7813fff640 1 -- 172.21.15.102:0/2405837070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77fc003630 msgr2=0x7f781c136c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:43.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.427+0000 7f7813fff640 1 --2- 172.21.15.102:0/2405837070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77fc003630 0x7f781c136c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:43.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.427+0000 7f7813fff640 1 -- 172.21.15.102:0/2405837070 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f781c098820 con 0x7f781c09dc00 2024-09-13T17:41:43.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.427+0000 7f7820a1d640 1 --2- 172.21.15.102:0/2405837070 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f781c09ffd0 0x7f781c130560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:41:43.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.427+0000 7f781bfff640 1 --2- 172.21.15.102:0/2405837070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77fc003630 0x7f781c136c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:41:43.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.427+0000 7f7813fff640 1 --2- 172.21.15.102:0/2405837070 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f781c09dc00 0x7f781c130020 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7804000c00 tx=0x7f7804008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:43.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.427+0000 7f7818ff9640 1 -- 172.21.15.102:0/2405837070 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780400f760 con 0x7f781c09dc00 2024-09-13T17:41:43.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.427+0000 7f7818ff9640 1 -- 172.21.15.102:0/2405837070 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f780400f8c0 con 0x7f781c09dc00 2024-09-13T17:41:43.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.427+0000 7f7822480640 1 -- 172.21.15.102:0/2405837070 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f781c131ba0 con 0x7f781c09dc00 2024-09-13T17:41:43.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.427+0000 7f7818ff9640 1 -- 172.21.15.102:0/2405837070 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780400fc80 con 0x7f781c09dc00 2024-09-13T17:41:43.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.427+0000 7f7822480640 1 -- 172.21.15.102:0/2405837070 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f781c134180 con 0x7f781c09dc00 2024-09-13T17:41:43.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.427+0000 7f7818ff9640 1 -- 172.21.15.102:0/2405837070 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f7804016670 con 0x7f781c09dc00 2024-09-13T17:41:43.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.427+0000 7f7818ff9640 1 -- 172.21.15.102:0/2405837070 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f7804020740 con 0x7f781c09dc00 2024-09-13T17:41:43.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.428+0000 7f7822480640 1 -- 172.21.15.102:0/2405837070 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f781c064600 con 0x7f781c09dc00 2024-09-13T17:41:43.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.431+0000 7f7818ff9640 1 -- 172.21.15.102:0/2405837070 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f7804025060 con 0x7f781c09dc00 2024-09-13T17:41:43.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.496+0000 7f7822480640 1 -- 172.21.15.102:0/2405837070 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f781c099a90 con 0x7f781c09dc00 2024-09-13T17:41:43.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.501+0000 7f7818ff9640 1 -- 172.21.15.102:0/2405837070 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f780402f090 con 0x7f781c09dc00 2024-09-13T17:41:43.506 INFO:teuthology.orchestra.run.smithi102.stderr:adjusted tunables profile to default 2024-09-13T17:41:43.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.502+0000 7f7822480640 1 -- 172.21.15.102:0/2405837070 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f781c09dc00 msgr2=0x7f781c130020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:43.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.502+0000 7f7822480640 1 --2- 172.21.15.102:0/2405837070 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f781c09dc00 0x7f781c130020 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7804000c00 tx=0x7f7804008040 comp rx=0 tx=0).stop 2024-09-13T17:41:43.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.502+0000 7f7822480640 1 -- 172.21.15.102:0/2405837070 shutdown_connections 2024-09-13T17:41:43.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.502+0000 7f7822480640 1 --2- 172.21.15.102:0/2405837070 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f781c09ffd0 0x7f781c130560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:43.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.502+0000 7f7822480640 1 --2- 172.21.15.102:0/2405837070 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f781c09dc00 0x7f781c130020 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:43.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.502+0000 7f7822480640 1 --2- 172.21.15.102:0/2405837070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77fc003630 0x7f781c136c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:43.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.502+0000 7f7822480640 1 -- 172.21.15.102:0/2405837070 >> 172.21.15.102:0/2405837070 conn(0x7f781c05e200 msgr2=0x7f781c09c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:43.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.503+0000 7f7822480640 1 -- 172.21.15.102:0/2405837070 shutdown_connections 2024-09-13T17:41:43.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.503+0000 7f7822480640 1 -- 172.21.15.102:0/2405837070 wait complete. 2024-09-13T17:41:43.516 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-13T17:41:43.513+0000 7f0b25e13ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T17:41:43.517 INFO:tasks.ceph:check_enable_crimson: False 2024-09-13T17:41:43.517 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-13T17:41:43.517 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-13T17:41:43.517 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-13T17:41:43.543 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-13T17:41:43.543 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-13T17:41:43.559 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-13T17:41:43.556+0000 7f0b25e13ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T17:41:43.584 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-13T17:41:43.584 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-13T17:41:43.611 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-13T17:41:43.608+0000 7f0b25e13ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T17:41:43.637 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-13T17:41:43.637 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-13T17:41:43.701 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new 16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e 0 2024-09-13T17:41:43.746 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-13T17:41:43.744+0000 7f0b25e13ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:41:43.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.832+0000 7fe035900640 1 Processor -- start 2024-09-13T17:41:43.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.832+0000 7fe035900640 1 -- start start 2024-09-13T17:41:43.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.832+0000 7fe035900640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe03012c6b0 0x7fe03012ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:43.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.833+0000 7fe035900640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe03009b4b0 0x7fe030134550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:43.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.833+0000 7fe035900640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe03012a2d0 con 0x7fe03009a990 2024-09-13T17:41:43.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.833+0000 7fe035900640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe030097da0 con 0x7fe03009b4b0 2024-09-13T17:41:43.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.833+0000 7fe035900640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe030098340 con 0x7fe03012c6b0 2024-09-13T17:41:43.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.833+0000 7fe02effd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe03012c6b0 0x7fe03012ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:43.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.833+0000 7fe02f7fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe03009b4b0 0x7fe030134550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:43.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.833+0000 7fe02effd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe03012c6b0 0x7fe03012ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:43146/0 (socket says 172.21.15.102:43146) 2024-09-13T17:41:43.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.833+0000 7fe02e7fc640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fe03009a990 0x7fe03009ada0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:41836/0 (socket says 172.21.15.102:41836) 2024-09-13T17:41:43.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.833+0000 7fe02effd640 1 -- 172.21.15.102:0/2263556935 learned_addr learned my addr 172.21.15.102:0/2263556935 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:41:43.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.833+0000 7fe02effd640 1 -- 172.21.15.102:0/2263556935 >> v2:172.21.15.102:3300/0 conn(0x7fe03009b4b0 msgr2=0x7fe030134550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:41:43.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.833+0000 7fe02effd640 1 --2- 172.21.15.102:0/2263556935 >> v2:172.21.15.102:3300/0 conn(0x7fe03009b4b0 0x7fe030134550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:43.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.833+0000 7fe02effd640 1 -- 172.21.15.102:0/2263556935 >> v1:172.21.15.102:6789/0 conn(0x7fe03009a990 legacy=0x7fe03009ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:43.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.833+0000 7fe02effd640 1 -- 172.21.15.102:0/2263556935 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03005e280 con 0x7fe03012c6b0 2024-09-13T17:41:43.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.833+0000 7fe02dffb640 1 -- 172.21.15.102:0/2263556935 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4045050489 0 0) 0x7fe03012a2d0 con 0x7fe03009a990 2024-09-13T17:41:43.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.833+0000 7fe02f7fe640 1 --2- 172.21.15.102:0/2263556935 >> v2:172.21.15.102:3300/0 conn(0x7fe03009b4b0 0x7fe030134550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:41:43.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.834+0000 7fe02effd640 1 --2- 172.21.15.102:0/2263556935 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe03012c6b0 0x7fe03012ca80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe024009910 tx=0x7fe0240091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44f635336c6608b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:43.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.834+0000 7fe02dffb640 1 -- 172.21.15.102:0/2263556935 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02402adb0 con 0x7fe03012c6b0 2024-09-13T17:41:43.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.834+0000 7fe02dffb640 1 -- 172.21.15.102:0/2263556935 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe024032440 con 0x7fe03012c6b0 2024-09-13T17:41:43.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.834+0000 7fe02dffb640 1 -- 172.21.15.102:0/2263556935 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0240327e0 con 0x7fe03012c6b0 2024-09-13T17:41:43.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.834+0000 7fe035900640 1 -- 172.21.15.102:0/2263556935 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe03012c6b0 msgr2=0x7fe03012ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:43.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.834+0000 7fe035900640 1 --2- 172.21.15.102:0/2263556935 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe03012c6b0 0x7fe03012ca80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe024009910 tx=0x7fe0240091d0 comp rx=0 tx=0).stop 2024-09-13T17:41:43.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.835+0000 7fe035900640 1 -- 172.21.15.102:0/2263556935 shutdown_connections 2024-09-13T17:41:43.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.835+0000 7fe035900640 1 --2- 172.21.15.102:0/2263556935 >> v2:172.21.15.102:3300/0 conn(0x7fe03009b4b0 0x7fe030134550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:43.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.835+0000 7fe035900640 1 --2- 172.21.15.102:0/2263556935 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe03012c6b0 0x7fe03012ca80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:43.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.835+0000 7fe035900640 1 -- 172.21.15.102:0/2263556935 >> 172.21.15.102:0/2263556935 conn(0x7fe03009fc40 msgr2=0x7fe03009ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:43.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.835+0000 7fe035900640 1 -- 172.21.15.102:0/2263556935 shutdown_connections 2024-09-13T17:41:43.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.835+0000 7fe035900640 1 -- 172.21.15.102:0/2263556935 wait complete. 2024-09-13T17:41:43.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.835+0000 7fe035900640 1 Processor -- start 2024-09-13T17:41:43.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.836+0000 7fe035900640 1 -- start start 2024-09-13T17:41:43.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.836+0000 7fe035900640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe03009a990 0x7fe030129590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:43.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.836+0000 7fe035900640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe03009b4b0 0x7fe030125fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:43.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.836+0000 7fe035900640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe03012c6b0 0x7fe030126510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:43.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.836+0000 7fe035900640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0300977b0 con 0x7fe03009a990 2024-09-13T17:41:43.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.836+0000 7fe035900640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe030128af0 con 0x7fe03012c6b0 2024-09-13T17:41:43.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.836+0000 7fe035900640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe030125cc0 con 0x7fe03009b4b0 2024-09-13T17:41:43.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.836+0000 7fe02effd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe03009a990 0x7fe030129590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:43.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.836+0000 7fe02e7fc640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe03009b4b0 0x7fe030125fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:43.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.836+0000 7fe02effd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe03009a990 0x7fe030129590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34192/0 (socket says 172.21.15.102:34192) 2024-09-13T17:41:43.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.836+0000 7fe02effd640 1 -- 172.21.15.102:0/4006814376 learned_addr learned my addr 172.21.15.102:0/4006814376 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:41:43.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.836+0000 7fe02f7fe640 1 --2- 172.21.15.102:0/4006814376 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe03012c6b0 0x7fe030126510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:43.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.836+0000 7fe02effd640 1 -- 172.21.15.102:0/4006814376 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe03009b4b0 msgr2=0x7fe030125fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:43.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.836+0000 7fe02effd640 1 --2- 172.21.15.102:0/4006814376 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe03009b4b0 0x7fe030125fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:43.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.836+0000 7fe02effd640 1 -- 172.21.15.102:0/4006814376 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe03012c6b0 msgr2=0x7fe030126510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:43.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.836+0000 7fe02effd640 1 --2- 172.21.15.102:0/4006814376 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe03012c6b0 0x7fe030126510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:43.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.836+0000 7fe02effd640 1 -- 172.21.15.102:0/4006814376 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe028009a20 con 0x7fe03009a990 2024-09-13T17:41:43.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.836+0000 7fe02f7fe640 1 --2- 172.21.15.102:0/4006814376 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe03012c6b0 0x7fe030126510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:41:43.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.837+0000 7fe02effd640 1 --2- 172.21.15.102:0/4006814376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe03009a990 0x7fe030129590 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe02402a6e0 tx=0x7fe024032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:43.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.837+0000 7fe016ffd640 1 -- 172.21.15.102:0/4006814376 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe024031bb0 con 0x7fe03009a990 2024-09-13T17:41:43.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.837+0000 7fe016ffd640 1 -- 172.21.15.102:0/4006814376 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe024031d30 con 0x7fe03009a990 2024-09-13T17:41:43.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.837+0000 7fe035900640 1 -- 172.21.15.102:0/4006814376 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe024009450 con 0x7fe03009a990 2024-09-13T17:41:43.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.837+0000 7fe016ffd640 1 -- 172.21.15.102:0/4006814376 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe024031ea0 con 0x7fe03009a990 2024-09-13T17:41:43.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.837+0000 7fe035900640 1 -- 172.21.15.102:0/4006814376 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe030126a50 con 0x7fe03009a990 2024-09-13T17:41:43.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.837+0000 7fe016ffd640 1 -- 172.21.15.102:0/4006814376 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fe024039040 con 0x7fe03009a990 2024-09-13T17:41:43.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.837+0000 7fe016ffd640 1 -- 172.21.15.102:0/4006814376 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7fe024046820 con 0x7fe03009a990 2024-09-13T17:41:43.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.838+0000 7fe035900640 1 -- 172.21.15.102:0/4006814376 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe030128f00 con 0x7fe03009a990 2024-09-13T17:41:43.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.840+0000 7fe016ffd640 1 -- 172.21.15.102:0/4006814376 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fe024042070 con 0x7fe03009a990 2024-09-13T17:41:43.859 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-13T17:41:43.857+0000 7f0b25e13ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T17:41:43.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.907+0000 7fe035900640 1 -- 172.21.15.102:0/4006814376 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd new", "uuid": "16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e", "id": 0} v 0) v1 -- 0x7fe030127690 con 0x7fe03009a990 2024-09-13T17:41:43.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.909+0000 7fe016ffd640 1 -- 172.21.15.102:0/4006814376 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe024037070 con 0x7fe03009a990 2024-09-13T17:41:43.912 INFO:teuthology.orchestra.run.smithi102.stdout:0 2024-09-13T17:41:43.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.910+0000 7fe035900640 1 -- 172.21.15.102:0/4006814376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe03009a990 msgr2=0x7fe030129590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:43.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.910+0000 7fe035900640 1 --2- 172.21.15.102:0/4006814376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe03009a990 0x7fe030129590 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe02402a6e0 tx=0x7fe024032000 comp rx=0 tx=0).stop 2024-09-13T17:41:43.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.910+0000 7fe035900640 1 -- 172.21.15.102:0/4006814376 shutdown_connections 2024-09-13T17:41:43.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.910+0000 7fe035900640 1 --2- 172.21.15.102:0/4006814376 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe03012c6b0 0x7fe030126510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:43.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.910+0000 7fe035900640 1 --2- 172.21.15.102:0/4006814376 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe03009b4b0 0x7fe030125fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:43.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.910+0000 7fe035900640 1 --2- 172.21.15.102:0/4006814376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe03009a990 0x7fe030129590 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:43.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.910+0000 7fe035900640 1 -- 172.21.15.102:0/4006814376 >> 172.21.15.102:0/4006814376 conn(0x7fe03009fc40 msgr2=0x7fe030096370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:43.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.910+0000 7fe035900640 1 -- 172.21.15.102:0/4006814376 shutdown_connections 2024-09-13T17:41:43.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:43.910+0000 7fe035900640 1 -- 172.21.15.102:0/4006814376 wait complete. 2024-09-13T17:41:43.923 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new d7b7e53f-89ed-4965-adae-314b40049a51 1 2024-09-13T17:41:44.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.054+0000 7f2684d9d640 1 Processor -- start 2024-09-13T17:41:44.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.055+0000 7f2684d9d640 1 -- start start 2024-09-13T17:41:44.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.055+0000 7f2684d9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f268006d640 0x7f268006e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:44.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.055+0000 7f2684d9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f268012d920 0x7f2680135560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:44.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.055+0000 7f2684d9d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f268009ff20 con 0x7f268006eba0 2024-09-13T17:41:44.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.055+0000 7f2684d9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f268006da10 con 0x7f268012d920 2024-09-13T17:41:44.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.055+0000 7f2684d9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f268006ceb0 con 0x7f268006d640 2024-09-13T17:41:44.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.055+0000 7f267e59c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f268006d640 0x7f268006e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:44.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.055+0000 7f267ed9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f268012d920 0x7f2680135560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:44.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.055+0000 7f267e59c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f268006d640 0x7f268006e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:40220/0 (socket says 172.21.15.102:40220) 2024-09-13T17:41:44.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.055+0000 7f267e59c640 1 -- 172.21.15.102:0/886587078 learned_addr learned my addr 172.21.15.102:0/886587078 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:41:44.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.055+0000 7f267dd9b640 1 --1- 172.21.15.102:0/886587078 >> v1:172.21.15.102:6789/0 conn(0x7f268006eba0 0x7f268012d210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:41846/0 (socket says 172.21.15.102:41846) 2024-09-13T17:41:44.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.056+0000 7f267e59c640 1 -- 172.21.15.102:0/886587078 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f268012d920 msgr2=0x7f2680135560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:41:44.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.056+0000 7f267e59c640 1 --2- 172.21.15.102:0/886587078 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f268012d920 0x7f2680135560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.056+0000 7f267e59c640 1 -- 172.21.15.102:0/886587078 >> v1:172.21.15.102:6789/0 conn(0x7f268006eba0 legacy=0x7f268012d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:44.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.056+0000 7f267e59c640 1 -- 172.21.15.102:0/886587078 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26800a02f0 con 0x7f268006d640 2024-09-13T17:41:44.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.056+0000 7f267e59c640 1 --2- 172.21.15.102:0/886587078 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f268006d640 0x7f268006e660 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f267000dda0 tx=0x7f267000d660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f49e7069c9727d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:44.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.056+0000 7f267d59a640 1 -- 172.21.15.102:0/886587078 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267000d2e0 con 0x7f268006d640 2024-09-13T17:41:44.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.056+0000 7f267d59a640 1 -- 172.21.15.102:0/886587078 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2670031050 con 0x7f268006d640 2024-09-13T17:41:44.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.057+0000 7f267d59a640 1 -- 172.21.15.102:0/886587078 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267000b680 con 0x7f268006d640 2024-09-13T17:41:44.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.057+0000 7f2684d9d640 1 -- 172.21.15.102:0/886587078 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f268006d640 msgr2=0x7f268006e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:44.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.057+0000 7f2684d9d640 1 --2- 172.21.15.102:0/886587078 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f268006d640 0x7f268006e660 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f267000dda0 tx=0x7f267000d660 comp rx=0 tx=0).stop 2024-09-13T17:41:44.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.057+0000 7f2684d9d640 1 -- 172.21.15.102:0/886587078 shutdown_connections 2024-09-13T17:41:44.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.057+0000 7f2684d9d640 1 --2- 172.21.15.102:0/886587078 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f268012d920 0x7f2680135560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.057+0000 7f2684d9d640 1 --2- 172.21.15.102:0/886587078 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f268006d640 0x7f268006e660 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.057+0000 7f2684d9d640 1 -- 172.21.15.102:0/886587078 >> 172.21.15.102:0/886587078 conn(0x7f2680067c00 msgr2=0x7f26800a08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:44.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.057+0000 7f2684d9d640 1 -- 172.21.15.102:0/886587078 shutdown_connections 2024-09-13T17:41:44.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.057+0000 7f2684d9d640 1 -- 172.21.15.102:0/886587078 wait complete. 2024-09-13T17:41:44.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.058+0000 7f2684d9d640 1 Processor -- start 2024-09-13T17:41:44.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.058+0000 7f2684d9d640 1 -- start start 2024-09-13T17:41:44.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.058+0000 7f2684d9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f268006d640 0x7f268012a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:44.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.058+0000 7f2684d9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f268006eba0 0x7f26801179d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:44.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.058+0000 7f2684d9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f268012d920 0x7f2680117f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:44.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.058+0000 7f2684d9d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2680119a40 con 0x7f268012d920 2024-09-13T17:41:44.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.058+0000 7f2684d9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2680119680 con 0x7f268006d640 2024-09-13T17:41:44.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.058+0000 7f2684d9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f268011a410 con 0x7f268006eba0 2024-09-13T17:41:44.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.059+0000 7f267e59c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f268006d640 0x7f268012a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:44.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.059+0000 7f267e59c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f268006d640 0x7f268012a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:43162/0 (socket says 172.21.15.102:43162) 2024-09-13T17:41:44.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.059+0000 7f267e59c640 1 -- 172.21.15.102:0/1939765872 learned_addr learned my addr 172.21.15.102:0/1939765872 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:41:44.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.059+0000 7f267ed9d640 1 --2- 172.21.15.102:0/1939765872 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f268012d920 0x7f2680117f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:44.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.059+0000 7f267dd9b640 1 --2- 172.21.15.102:0/1939765872 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f268006eba0 0x7f26801179d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:44.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.059+0000 7f267ed9d640 1 -- 172.21.15.102:0/1939765872 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f268006eba0 msgr2=0x7f26801179d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:44.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.059+0000 7f267ed9d640 1 --2- 172.21.15.102:0/1939765872 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f268006eba0 0x7f26801179d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.059+0000 7f267ed9d640 1 -- 172.21.15.102:0/1939765872 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f268006d640 msgr2=0x7f268012a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:44.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.059+0000 7f267ed9d640 1 --2- 172.21.15.102:0/1939765872 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f268006d640 0x7f268012a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.059+0000 7f267ed9d640 1 -- 172.21.15.102:0/1939765872 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f267000d900 con 0x7f268012d920 2024-09-13T17:41:44.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.059+0000 7f267e59c640 1 --2- 172.21.15.102:0/1939765872 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f268006d640 0x7f268012a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:41:44.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.059+0000 7f267dd9b640 1 --2- 172.21.15.102:0/1939765872 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f268006eba0 0x7f26801179d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:41:44.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.059+0000 7f267ed9d640 1 --2- 172.21.15.102:0/1939765872 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f268012d920 0x7f2680117f10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2678009e50 tx=0x7f267800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:44.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.059+0000 7f266e7fc640 1 -- 172.21.15.102:0/1939765872 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267800cd40 con 0x7f268012d920 2024-09-13T17:41:44.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.059+0000 7f266e7fc640 1 -- 172.21.15.102:0/1939765872 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267800f040 con 0x7f268012d920 2024-09-13T17:41:44.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.059+0000 7f2684d9d640 1 -- 172.21.15.102:0/1939765872 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f268011ae70 con 0x7f268012d920 2024-09-13T17:41:44.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.059+0000 7f266e7fc640 1 -- 172.21.15.102:0/1939765872 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26780146d0 con 0x7f268012d920 2024-09-13T17:41:44.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.059+0000 7f2684d9d640 1 -- 172.21.15.102:0/1939765872 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2680119c80 con 0x7f268012d920 2024-09-13T17:41:44.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.059+0000 7f266e7fc640 1 -- 172.21.15.102:0/1939765872 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f2678014cd0 con 0x7f268012d920 2024-09-13T17:41:44.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.059+0000 7f266e7fc640 1 -- 172.21.15.102:0/1939765872 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7f267801de20 con 0x7f268012d920 2024-09-13T17:41:44.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.060+0000 7f2684d9d640 1 -- 172.21.15.102:0/1939765872 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2680064600 con 0x7f268012d920 2024-09-13T17:41:44.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.062+0000 7f266e7fc640 1 -- 172.21.15.102:0/1939765872 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f267803b7a0 con 0x7f268012d920 2024-09-13T17:41:44.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.128+0000 7f2684d9d640 1 -- 172.21.15.102:0/1939765872 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d7b7e53f-89ed-4965-adae-314b40049a51", "id": 1} v 0) v1 -- 0x7f268009af10 con 0x7f268012d920 2024-09-13T17:41:44.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.133+0000 7f266e7fc640 1 -- 172.21.15.102:0/1939765872 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d7b7e53f-89ed-4965-adae-314b40049a51", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f267802a070 con 0x7f268012d920 2024-09-13T17:41:44.136 INFO:teuthology.orchestra.run.smithi102.stdout:1 2024-09-13T17:41:44.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.134+0000 7f2684d9d640 1 -- 172.21.15.102:0/1939765872 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f268012d920 msgr2=0x7f2680117f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:44.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.134+0000 7f2684d9d640 1 --2- 172.21.15.102:0/1939765872 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f268012d920 0x7f2680117f10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2678009e50 tx=0x7f267800b040 comp rx=0 tx=0).stop 2024-09-13T17:41:44.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.134+0000 7f2684d9d640 1 -- 172.21.15.102:0/1939765872 shutdown_connections 2024-09-13T17:41:44.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.134+0000 7f2684d9d640 1 --2- 172.21.15.102:0/1939765872 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f268012d920 0x7f2680117f10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.134+0000 7f2684d9d640 1 --2- 172.21.15.102:0/1939765872 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f268006eba0 0x7f26801179d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.134+0000 7f2684d9d640 1 --2- 172.21.15.102:0/1939765872 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f268006d640 0x7f268012a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.134+0000 7f2684d9d640 1 -- 172.21.15.102:0/1939765872 >> 172.21.15.102:0/1939765872 conn(0x7f2680067c00 msgr2=0x7f268009d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:44.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.134+0000 7f2684d9d640 1 -- 172.21.15.102:0/1939765872 shutdown_connections 2024-09-13T17:41:44.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.134+0000 7f2684d9d640 1 -- 172.21.15.102:0/1939765872 wait complete. 2024-09-13T17:41:44.146 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new 7c748ced-47e9-4007-aabe-21d4d2a9ed75 2 2024-09-13T17:41:44.163 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-13T17:41:44.160+0000 7f0b25e13ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T17:41:44.236 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-13T17:41:44.233+0000 7f0b25e13ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T17:41:44.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.237+0000 7f8b0b4d4640 1 Processor -- start 2024-09-13T17:41:44.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.237+0000 7f8b0b4d4640 1 -- start start 2024-09-13T17:41:44.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.237+0000 7f8b0b4d4640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8b04098a10 0x7f8b04098330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:44.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.237+0000 7f8b0b4d4640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8b0409ae60 0x7f8b0412c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:44.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.237+0000 7f8b0b4d4640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8b0409b250 0x7f8b0412c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:44.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.237+0000 7f8b0b4d4640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b040975e0 con 0x7f8b04098a10 2024-09-13T17:41:44.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.237+0000 7f8b0b4d4640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0409bca0 con 0x7f8b0409ae60 2024-09-13T17:41:44.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.237+0000 7f8b0b4d4640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b04099540 con 0x7f8b0409b250 2024-09-13T17:41:44.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.237+0000 7f8b09270640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8b04098a10 0x7f8b04098330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:44.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.237+0000 7f8b09a71640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8b0409b250 0x7f8b0412c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:44.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.237+0000 7f8b09270640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8b04098a10 0x7f8b04098330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34206/0 (socket says 172.21.15.102:34206) 2024-09-13T17:41:44.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.237+0000 7f8b09270640 1 -- 172.21.15.102:0/4130053411 learned_addr learned my addr 172.21.15.102:0/4130053411 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:41:44.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.237+0000 7f8b08a6f640 1 --2- 172.21.15.102:0/4130053411 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8b0409ae60 0x7f8b0412c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:44.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.238+0000 7f8b09270640 1 -- 172.21.15.102:0/4130053411 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8b0409b250 msgr2=0x7f8b0412c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:41:44.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.238+0000 7f8b09270640 1 --2- 172.21.15.102:0/4130053411 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8b0409b250 0x7f8b0412c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.238+0000 7f8b09270640 1 -- 172.21.15.102:0/4130053411 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8b0409ae60 msgr2=0x7f8b0412c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:41:44.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.238+0000 7f8b09270640 1 --2- 172.21.15.102:0/4130053411 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8b0409ae60 0x7f8b0412c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.238+0000 7f8b09270640 1 -- 172.21.15.102:0/4130053411 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b04097890 con 0x7f8b04098a10 2024-09-13T17:41:44.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.238+0000 7f8b08a6f640 1 --2- 172.21.15.102:0/4130053411 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8b0409ae60 0x7f8b0412c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:41:44.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.238+0000 7f8b09270640 1 --2- 172.21.15.102:0/4130053411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b04098a10 0x7f8b04098330 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000bbc0 tx=0x7f8b0000b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c72c156321b33468 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:44.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.239+0000 7f8afbfff640 1 -- 172.21.15.102:0/4130053411 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b000324f0 con 0x7f8b04098a10 2024-09-13T17:41:44.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.239+0000 7f8afbfff640 1 -- 172.21.15.102:0/4130053411 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b00032650 con 0x7f8b04098a10 2024-09-13T17:41:44.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.239+0000 7f8afbfff640 1 -- 172.21.15.102:0/4130053411 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b000329f0 con 0x7f8b04098a10 2024-09-13T17:41:44.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.239+0000 7f8b0b4d4640 1 -- 172.21.15.102:0/4130053411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b04098a10 msgr2=0x7f8b04098330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:44.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.239+0000 7f8b0b4d4640 1 --2- 172.21.15.102:0/4130053411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b04098a10 0x7f8b04098330 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000bbc0 tx=0x7f8b0000b460 comp rx=0 tx=0).stop 2024-09-13T17:41:44.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.239+0000 7f8b0b4d4640 1 -- 172.21.15.102:0/4130053411 shutdown_connections 2024-09-13T17:41:44.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.239+0000 7f8b0b4d4640 1 --2- 172.21.15.102:0/4130053411 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8b0409b250 0x7f8b0412c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.239+0000 7f8b0b4d4640 1 --2- 172.21.15.102:0/4130053411 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8b0409ae60 0x7f8b0412c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.239+0000 7f8b0b4d4640 1 --2- 172.21.15.102:0/4130053411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b04098a10 0x7f8b04098330 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.239+0000 7f8b0b4d4640 1 -- 172.21.15.102:0/4130053411 >> 172.21.15.102:0/4130053411 conn(0x7f8b0405e610 msgr2=0x7f8b0409f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:44.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.239+0000 7f8b0b4d4640 1 -- 172.21.15.102:0/4130053411 shutdown_connections 2024-09-13T17:41:44.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.239+0000 7f8b0b4d4640 1 -- 172.21.15.102:0/4130053411 wait complete. 2024-09-13T17:41:44.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.240+0000 7f8b0b4d4640 1 Processor -- start 2024-09-13T17:41:44.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.240+0000 7f8b0b4d4640 1 -- start start 2024-09-13T17:41:44.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.241+0000 7f8b0b4d4640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b04098a10 0x7f8b04129890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:44.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.241+0000 7f8b0b4d4640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8b0409ae60 0x7f8b0411a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:44.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.241+0000 7f8b0b4d4640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8b0409b250 0x7f8b0411a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:44.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.241+0000 7f8b0b4d4640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0411b9a0 con 0x7f8b04098a10 2024-09-13T17:41:44.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.241+0000 7f8b0b4d4640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0411b460 con 0x7f8b0409ae60 2024-09-13T17:41:44.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.241+0000 7f8b0b4d4640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b04129dd0 con 0x7f8b0409b250 2024-09-13T17:41:44.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.241+0000 7f8b09270640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b04098a10 0x7f8b04129890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:44.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.241+0000 7f8b08a6f640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8b0409ae60 0x7f8b0411a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:44.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.241+0000 7f8b09a71640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8b0409b250 0x7f8b0411a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:44.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.241+0000 7f8b08a6f640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8b0409ae60 0x7f8b0411a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:43180/0 (socket says 172.21.15.102:43180) 2024-09-13T17:41:44.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.241+0000 7f8b08a6f640 1 -- 172.21.15.102:0/74163398 learned_addr learned my addr 172.21.15.102:0/74163398 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:41:44.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.241+0000 7f8b09a71640 1 -- 172.21.15.102:0/74163398 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8b0409ae60 msgr2=0x7f8b0411a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:44.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.241+0000 7f8b09a71640 1 --2- 172.21.15.102:0/74163398 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8b0409ae60 0x7f8b0411a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.241+0000 7f8b09a71640 1 -- 172.21.15.102:0/74163398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b04098a10 msgr2=0x7f8b04129890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:44.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.241+0000 7f8b09a71640 1 --2- 172.21.15.102:0/74163398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b04098a10 0x7f8b04129890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.241+0000 7f8b09a71640 1 -- 172.21.15.102:0/74163398 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af4009390 con 0x7f8b0409b250 2024-09-13T17:41:44.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.241+0000 7f8b08a6f640 1 --2- 172.21.15.102:0/74163398 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8b0409ae60 0x7f8b0411a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:41:44.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.241+0000 7f8b09270640 1 --2- 172.21.15.102:0/74163398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b04098a10 0x7f8b04129890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:41:44.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.241+0000 7f8b09a71640 1 --2- 172.21.15.102:0/74163398 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8b0409b250 0x7f8b0411a7e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8afc009e50 tx=0x7f8afc00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:44.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.242+0000 7f8af97fa640 1 -- 172.21.15.102:0/74163398 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc00cd40 con 0x7f8b0409b250 2024-09-13T17:41:44.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.242+0000 7f8af97fa640 1 -- 172.21.15.102:0/74163398 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8afc00f040 con 0x7f8b0409b250 2024-09-13T17:41:44.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.242+0000 7f8af97fa640 1 -- 172.21.15.102:0/74163398 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc00cea0 con 0x7f8b0409b250 2024-09-13T17:41:44.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.242+0000 7f8b0b4d4640 1 -- 172.21.15.102:0/74163398 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b0000b700 con 0x7f8b0409b250 2024-09-13T17:41:44.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.242+0000 7f8b0b4d4640 1 -- 172.21.15.102:0/74163398 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b0411d090 con 0x7f8b0409b250 2024-09-13T17:41:44.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.242+0000 7f8af97fa640 1 -- 172.21.15.102:0/74163398 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f8afc014b70 con 0x7f8b0409b250 2024-09-13T17:41:44.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.242+0000 7f8af97fa640 1 -- 172.21.15.102:0/74163398 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f8afc01ddd0 con 0x7f8b0409b250 2024-09-13T17:41:44.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.243+0000 7f8b0b4d4640 1 -- 172.21.15.102:0/74163398 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b04064600 con 0x7f8b0409b250 2024-09-13T17:41:44.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.245+0000 7f8af97fa640 1 -- 172.21.15.102:0/74163398 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f8afc03b7a0 con 0x7f8b0409b250 2024-09-13T17:41:44.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.312+0000 7f8b0b4d4640 1 -- 172.21.15.102:0/74163398 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd new", "uuid": "7c748ced-47e9-4007-aabe-21d4d2a9ed75", "id": 2} v 0) v1 -- 0x7f8b0411bbe0 con 0x7f8b0409b250 2024-09-13T17:41:44.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.320+0000 7f8af97fa640 1 -- 172.21.15.102:0/74163398 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7c748ced-47e9-4007-aabe-21d4d2a9ed75", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8afc02a070 con 0x7f8b0409b250 2024-09-13T17:41:44.323 INFO:teuthology.orchestra.run.smithi102.stdout:2 2024-09-13T17:41:44.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.321+0000 7f8b0b4d4640 1 -- 172.21.15.102:0/74163398 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8b0409b250 msgr2=0x7f8b0411a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:44.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.321+0000 7f8b0b4d4640 1 --2- 172.21.15.102:0/74163398 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8b0409b250 0x7f8b0411a7e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8afc009e50 tx=0x7f8afc00b040 comp rx=0 tx=0).stop 2024-09-13T17:41:44.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.321+0000 7f8b0b4d4640 1 -- 172.21.15.102:0/74163398 shutdown_connections 2024-09-13T17:41:44.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.321+0000 7f8b0b4d4640 1 --2- 172.21.15.102:0/74163398 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8b0409b250 0x7f8b0411a7e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.321+0000 7f8b0b4d4640 1 --2- 172.21.15.102:0/74163398 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8b0409ae60 0x7f8b0411a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.321+0000 7f8b0b4d4640 1 --2- 172.21.15.102:0/74163398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b04098a10 0x7f8b04129890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.321+0000 7f8b0b4d4640 1 -- 172.21.15.102:0/74163398 >> 172.21.15.102:0/74163398 conn(0x7f8b0405e610 msgr2=0x7f8b04099c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:44.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.321+0000 7f8b0b4d4640 1 -- 172.21.15.102:0/74163398 shutdown_connections 2024-09-13T17:41:44.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.321+0000 7f8b0b4d4640 1 -- 172.21.15.102:0/74163398 wait complete. 2024-09-13T17:41:44.334 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new 743a5d12-dd6c-41c6-a461-dd62b868d710 3 2024-09-13T17:41:44.361 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-13T17:41:44.358+0000 7f0b25e13ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T17:41:44.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.453+0000 7feec259d640 1 Processor -- start 2024-09-13T17:41:44.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.453+0000 7feec259d640 1 -- start start 2024-09-13T17:41:44.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.453+0000 7feec259d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7feebc098a10 0x7feebc098330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:44.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.453+0000 7feec259d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feebc09b280 0x7feebc12c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:44.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.453+0000 7feec259d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feebc0975e0 con 0x7feebc09ae60 2024-09-13T17:41:44.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.453+0000 7feec259d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feebc09bca0 con 0x7feebc098a10 2024-09-13T17:41:44.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.453+0000 7feec259d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feebc099540 con 0x7feebc09b280 2024-09-13T17:41:44.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.454+0000 7feebbfff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7feebc098a10 0x7feebc098330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:44.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.454+0000 7feec0b3a640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feebc09b280 0x7feebc12c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:44.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.454+0000 7feebbfff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7feebc098a10 0x7feebc098330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34230/0 (socket says 172.21.15.102:34230) 2024-09-13T17:41:44.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.454+0000 7feebb7fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7feebc09ae60 0x7feebc12c160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:41860/0 (socket says 172.21.15.102:41860) 2024-09-13T17:41:44.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.454+0000 7feec0b3a640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feebc09b280 0x7feebc12c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:40260/0 (socket says 172.21.15.102:40260) 2024-09-13T17:41:44.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.454+0000 7feebbfff640 1 -- 172.21.15.102:0/2754737219 learned_addr learned my addr 172.21.15.102:0/2754737219 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:41:44.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.454+0000 7feebbfff640 1 -- 172.21.15.102:0/2754737219 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feebc09b280 msgr2=0x7feebc12c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:41:44.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.454+0000 7feebbfff640 1 --2- 172.21.15.102:0/2754737219 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feebc09b280 0x7feebc12c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.454+0000 7feebbfff640 1 -- 172.21.15.102:0/2754737219 >> v1:172.21.15.102:6789/0 conn(0x7feebc09ae60 legacy=0x7feebc12c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:44.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.454+0000 7feebaffd640 1 -- 172.21.15.102:0/2754737219 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 715073569 0 0) 0x7feebc0975e0 con 0x7feebc09ae60 2024-09-13T17:41:44.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.454+0000 7feebbfff640 1 -- 172.21.15.102:0/2754737219 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feebc097890 con 0x7feebc098a10 2024-09-13T17:41:44.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.454+0000 7feec0b3a640 1 --2- 172.21.15.102:0/2754737219 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feebc09b280 0x7feebc12c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:41:44.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.455+0000 7feebbfff640 1 --2- 172.21.15.102:0/2754737219 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feebc098a10 0x7feebc098330 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7feea4009910 tx=0x7feea40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c271572036fe57 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:44.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.455+0000 7feebaffd640 1 -- 172.21.15.102:0/2754737219 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea402ad20 con 0x7feebc098a10 2024-09-13T17:41:44.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.455+0000 7feebaffd640 1 -- 172.21.15.102:0/2754737219 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea402ae80 con 0x7feebc098a10 2024-09-13T17:41:44.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.455+0000 7feebaffd640 1 -- 172.21.15.102:0/2754737219 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea4032670 con 0x7feebc098a10 2024-09-13T17:41:44.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.455+0000 7feec259d640 1 -- 172.21.15.102:0/2754737219 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feebc098a10 msgr2=0x7feebc098330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:44.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.455+0000 7feec259d640 1 --2- 172.21.15.102:0/2754737219 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feebc098a10 0x7feebc098330 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7feea4009910 tx=0x7feea40091d0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.455+0000 7feec259d640 1 -- 172.21.15.102:0/2754737219 shutdown_connections 2024-09-13T17:41:44.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.455+0000 7feec259d640 1 --2- 172.21.15.102:0/2754737219 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feebc09b280 0x7feebc12c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.455+0000 7feec259d640 1 --2- 172.21.15.102:0/2754737219 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feebc098a10 0x7feebc098330 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.455+0000 7feec259d640 1 -- 172.21.15.102:0/2754737219 >> 172.21.15.102:0/2754737219 conn(0x7feebc05e610 msgr2=0x7feebc09f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:44.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.455+0000 7feec259d640 1 -- 172.21.15.102:0/2754737219 shutdown_connections 2024-09-13T17:41:44.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.455+0000 7feec259d640 1 -- 172.21.15.102:0/2754737219 wait complete. 2024-09-13T17:41:44.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.456+0000 7feec259d640 1 Processor -- start 2024-09-13T17:41:44.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.456+0000 7feec259d640 1 -- start start 2024-09-13T17:41:44.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.457+0000 7feec259d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feebc098a10 0x7feebc118bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:44.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.457+0000 7feec259d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feebc09ae60 0x7feebc119110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:44.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.457+0000 7feec259d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feebc09b280 0x7feebc119650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:44.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.457+0000 7feec259d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc11aac0 con 0x7feebc098a10 2024-09-13T17:41:44.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.457+0000 7feec259d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feebc11ce50 con 0x7feebc09ae60 2024-09-13T17:41:44.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.457+0000 7feec259d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feebc11c1b0 con 0x7feebc09b280 2024-09-13T17:41:44.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.457+0000 7feebbfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feebc098a10 0x7feebc118bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:44.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.457+0000 7feebbfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feebc098a10 0x7feebc118bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34234/0 (socket says 172.21.15.102:34234) 2024-09-13T17:41:44.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.457+0000 7feebbfff640 1 -- 172.21.15.102:0/1083127355 learned_addr learned my addr 172.21.15.102:0/1083127355 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:41:44.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.457+0000 7feec0b3a640 1 --2- 172.21.15.102:0/1083127355 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feebc09b280 0x7feebc119650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:44.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.457+0000 7feebb7fe640 1 --2- 172.21.15.102:0/1083127355 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feebc09ae60 0x7feebc119110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:44.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.457+0000 7feebbfff640 1 -- 172.21.15.102:0/1083127355 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feebc09b280 msgr2=0x7feebc119650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:44.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.457+0000 7feebbfff640 1 --2- 172.21.15.102:0/1083127355 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feebc09b280 0x7feebc119650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.457+0000 7feebbfff640 1 -- 172.21.15.102:0/1083127355 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feebc09ae60 msgr2=0x7feebc119110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:44.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.457+0000 7feebbfff640 1 --2- 172.21.15.102:0/1083127355 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feebc09ae60 0x7feebc119110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.457+0000 7feebbfff640 1 -- 172.21.15.102:0/1083127355 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb4009a20 con 0x7feebc098a10 2024-09-13T17:41:44.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.457+0000 7feec0b3a640 1 --2- 172.21.15.102:0/1083127355 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feebc09b280 0x7feebc119650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:41:44.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.457+0000 7feebbfff640 1 --2- 172.21.15.102:0/1083127355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feebc098a10 0x7feebc118bd0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7feea4009970 tx=0x7feea4032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:44.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.457+0000 7feeb3fff640 1 -- 172.21.15.102:0/1083127355 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea403b6e0 con 0x7feebc098a10 2024-09-13T17:41:44.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.457+0000 7feeb3fff640 1 -- 172.21.15.102:0/1083127355 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea403b860 con 0x7feebc098a10 2024-09-13T17:41:44.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.457+0000 7feeb3fff640 1 -- 172.21.15.102:0/1083127355 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea403bc60 con 0x7feebc098a10 2024-09-13T17:41:44.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.458+0000 7feec259d640 1 -- 172.21.15.102:0/1083127355 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feea4009450 con 0x7feebc098a10 2024-09-13T17:41:44.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.458+0000 7feec259d640 1 -- 172.21.15.102:0/1083127355 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feebc119fb0 con 0x7feebc098a10 2024-09-13T17:41:44.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.458+0000 7feeb3fff640 1 -- 172.21.15.102:0/1083127355 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7feea4043030 con 0x7feebc098a10 2024-09-13T17:41:44.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.458+0000 7feeb3fff640 1 -- 172.21.15.102:0/1083127355 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7feea4041070 con 0x7feebc098a10 2024-09-13T17:41:44.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.458+0000 7feec259d640 1 -- 172.21.15.102:0/1083127355 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feebc11c2f0 con 0x7feebc098a10 2024-09-13T17:41:44.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.460+0000 7feeb3fff640 1 -- 172.21.15.102:0/1083127355 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7feebc11c2f0 con 0x7feebc098a10 2024-09-13T17:41:44.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.528+0000 7feec259d640 1 -- 172.21.15.102:0/1083127355 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd new", "uuid": "743a5d12-dd6c-41c6-a461-dd62b868d710", "id": 3} v 0) v1 -- 0x7feebc064600 con 0x7feebc098a10 2024-09-13T17:41:44.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.533+0000 7feeb3fff640 1 -- 172.21.15.102:0/1083127355 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "743a5d12-dd6c-41c6-a461-dd62b868d710", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7feea40550b0 con 0x7feebc098a10 2024-09-13T17:41:44.535 INFO:teuthology.orchestra.run.smithi102.stdout:3 2024-09-13T17:41:44.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.534+0000 7feec259d640 1 -- 172.21.15.102:0/1083127355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feebc098a10 msgr2=0x7feebc118bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:44.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.534+0000 7feec259d640 1 --2- 172.21.15.102:0/1083127355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feebc098a10 0x7feebc118bd0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7feea4009970 tx=0x7feea4032000 comp rx=0 tx=0).stop 2024-09-13T17:41:44.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.534+0000 7feec259d640 1 -- 172.21.15.102:0/1083127355 shutdown_connections 2024-09-13T17:41:44.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.534+0000 7feec259d640 1 --2- 172.21.15.102:0/1083127355 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feebc09b280 0x7feebc119650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.534+0000 7feec259d640 1 --2- 172.21.15.102:0/1083127355 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feebc09ae60 0x7feebc119110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.534+0000 7feec259d640 1 --2- 172.21.15.102:0/1083127355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feebc098a10 0x7feebc118bd0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.534+0000 7feec259d640 1 -- 172.21.15.102:0/1083127355 >> 172.21.15.102:0/1083127355 conn(0x7feebc05e610 msgr2=0x7feebc12b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:44.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.534+0000 7feec259d640 1 -- 172.21.15.102:0/1083127355 shutdown_connections 2024-09-13T17:41:44.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.534+0000 7feec259d640 1 -- 172.21.15.102:0/1083127355 wait complete. 2024-09-13T17:41:44.547 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-13T17:41:44.547 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-13T17:41:44.550 INFO:tasks.ceph.osd.0:Started 2024-09-13T17:41:44.550 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-13T17:41:44.551 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-13T17:41:44.554 INFO:tasks.ceph.osd.1:Started 2024-09-13T17:41:44.554 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-13T17:41:44.554 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-13T17:41:44.557 INFO:tasks.ceph.osd.2:Started 2024-09-13T17:41:44.557 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-13T17:41:44.557 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-13T17:41:44.559 INFO:tasks.ceph.osd.3:Started 2024-09-13T17:41:44.560 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:41:44.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.687+0000 7f641e500640 1 Processor -- start 2024-09-13T17:41:44.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.687+0000 7f641e500640 1 -- start start 2024-09-13T17:41:44.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.687+0000 7f641e500640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6418097670 0x7f641809a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:44.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.687+0000 7f641e500640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f641809c1e0 0x7f641809ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:44.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.687+0000 7f641e500640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f641809b290 0x7f6418135fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:44.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.687+0000 7f641e500640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6418096f00 con 0x7f641809b290 2024-09-13T17:41:44.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.687+0000 7f641e500640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64180992f0 con 0x7f641809c1e0 2024-09-13T17:41:44.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.687+0000 7f641e500640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f641809c5b0 con 0x7f6418097670 2024-09-13T17:41:44.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.687+0000 7f6417fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6418097670 0x7f641809a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:44.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.687+0000 7f641ca9d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f641809b290 0x7f6418135fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:44.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.688+0000 7f64177fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f641809c1e0 0x7f641809ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:44.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.688+0000 7f6417fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6418097670 0x7f641809a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:40280/0 (socket says 172.21.15.102:40280) 2024-09-13T17:41:44.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.688+0000 7f641ca9d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f641809b290 0x7f6418135fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34266/0 (socket says 172.21.15.102:34266) 2024-09-13T17:41:44.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.688+0000 7f6417fff640 1 -- 172.21.15.102:0/1523652085 learned_addr learned my addr 172.21.15.102:0/1523652085 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:41:44.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.688+0000 7f64177fe640 1 -- 172.21.15.102:0/1523652085 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6418097670 msgr2=0x7f641809a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:41:44.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.688+0000 7f64177fe640 1 --2- 172.21.15.102:0/1523652085 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6418097670 0x7f641809a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.688+0000 7f64177fe640 1 -- 172.21.15.102:0/1523652085 >> v2:172.21.15.102:3300/0 conn(0x7f641809b290 msgr2=0x7f6418135fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:41:44.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.688+0000 7f64177fe640 1 --2- 172.21.15.102:0/1523652085 >> v2:172.21.15.102:3300/0 conn(0x7f641809b290 0x7f6418135fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.688+0000 7f64177fe640 1 -- 172.21.15.102:0/1523652085 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f641805e2e0 con 0x7f641809c1e0 2024-09-13T17:41:44.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.688+0000 7f6417fff640 1 --2- 172.21.15.102:0/1523652085 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6418097670 0x7f641809a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:41:44.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.688+0000 7f64177fe640 1 --2- 172.21.15.102:0/1523652085 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f641809c1e0 0x7f641809ad50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f640c009850 tx=0x7f640c0090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4f7655cc868731 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:44.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.689+0000 7f6416ffd640 1 -- 172.21.15.102:0/1523652085 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c02ac50 con 0x7f641809c1e0 2024-09-13T17:41:44.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.689+0000 7f6416ffd640 1 -- 172.21.15.102:0/1523652085 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f640c02adb0 con 0x7f641809c1e0 2024-09-13T17:41:44.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.689+0000 7f6416ffd640 1 -- 172.21.15.102:0/1523652085 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c032670 con 0x7f641809c1e0 2024-09-13T17:41:44.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.689+0000 7f641e500640 1 -- 172.21.15.102:0/1523652085 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f641809c1e0 msgr2=0x7f641809ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:44.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.689+0000 7f641e500640 1 --2- 172.21.15.102:0/1523652085 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f641809c1e0 0x7f641809ad50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f640c009850 tx=0x7f640c0090f0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.689+0000 7f641e500640 1 -- 172.21.15.102:0/1523652085 shutdown_connections 2024-09-13T17:41:44.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.689+0000 7f641e500640 1 --2- 172.21.15.102:0/1523652085 >> v2:172.21.15.102:3300/0 conn(0x7f641809b290 0x7f6418135fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.689+0000 7f641e500640 1 --2- 172.21.15.102:0/1523652085 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f641809c1e0 0x7f641809ad50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.689+0000 7f641e500640 1 --2- 172.21.15.102:0/1523652085 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6418097670 0x7f641809a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.689+0000 7f641e500640 1 -- 172.21.15.102:0/1523652085 >> 172.21.15.102:0/1523652085 conn(0x7f641809f8b0 msgr2=0x7f641809b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:44.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.689+0000 7f641e500640 1 -- 172.21.15.102:0/1523652085 shutdown_connections 2024-09-13T17:41:44.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.689+0000 7f641e500640 1 -- 172.21.15.102:0/1523652085 wait complete. 2024-09-13T17:41:44.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.690+0000 7f641e500640 1 Processor -- start 2024-09-13T17:41:44.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.690+0000 7f641e500640 1 -- start start 2024-09-13T17:41:44.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.691+0000 7f641e500640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6418097670 0x7f64180906d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:44.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.691+0000 7f641e500640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f641809b290 0x7f6418090c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:44.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.691+0000 7f641e500640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f641809c1e0 0x7f6418091150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:44.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.691+0000 7f641e500640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6418091b40 con 0x7f641809c1e0 2024-09-13T17:41:44.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.691+0000 7f641e500640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f641809daa0 con 0x7f6418097670 2024-09-13T17:41:44.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.691+0000 7f641e500640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6418096520 con 0x7f641809b290 2024-09-13T17:41:44.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.691+0000 7f6417fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6418097670 0x7f64180906d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:44.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.691+0000 7f64177fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f641809b290 0x7f6418090c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:44.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.691+0000 7f6417fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6418097670 0x7f64180906d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:43224/0 (socket says 172.21.15.102:43224) 2024-09-13T17:41:44.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.691+0000 7f6417fff640 1 -- 172.21.15.102:0/1667929008 learned_addr learned my addr 172.21.15.102:0/1667929008 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:41:44.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.691+0000 7f641ca9d640 1 --2- 172.21.15.102:0/1667929008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f641809c1e0 0x7f6418091150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:44.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.691+0000 7f6417fff640 1 -- 172.21.15.102:0/1667929008 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f641809b290 msgr2=0x7f6418090c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:44.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.691+0000 7f6417fff640 1 --2- 172.21.15.102:0/1667929008 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f641809b290 0x7f6418090c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.691+0000 7f6417fff640 1 -- 172.21.15.102:0/1667929008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f641809c1e0 msgr2=0x7f6418091150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:44.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.691+0000 7f6417fff640 1 --2- 172.21.15.102:0/1667929008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f641809c1e0 0x7f6418091150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.691+0000 7f6417fff640 1 -- 172.21.15.102:0/1667929008 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6408009450 con 0x7f6418097670 2024-09-13T17:41:44.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.691+0000 7f64177fe640 1 --2- 172.21.15.102:0/1667929008 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f641809b290 0x7f6418090c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:41:44.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.691+0000 7f641ca9d640 1 --2- 172.21.15.102:0/1667929008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f641809c1e0 0x7f6418091150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:41:44.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.691+0000 7f6417fff640 1 --2- 172.21.15.102:0/1667929008 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6418097670 0x7f64180906d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f640800ea30 tx=0x7f64080091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:44.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.691+0000 7f63fffff640 1 -- 172.21.15.102:0/1667929008 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640800ca10 con 0x7f6418097670 2024-09-13T17:41:44.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.691+0000 7f63fffff640 1 -- 172.21.15.102:0/1667929008 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f640800cb70 con 0x7f6418097670 2024-09-13T17:41:44.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.691+0000 7f641e500640 1 -- 172.21.15.102:0/1667929008 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f640c009390 con 0x7f6418097670 2024-09-13T17:41:44.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.691+0000 7f63fffff640 1 -- 172.21.15.102:0/1667929008 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6408013470 con 0x7f6418097670 2024-09-13T17:41:44.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.692+0000 7f641e500640 1 -- 172.21.15.102:0/1667929008 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6418096960 con 0x7f6418097670 2024-09-13T17:41:44.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.692+0000 7f63fffff640 1 -- 172.21.15.102:0/1667929008 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f6408013820 con 0x7f6418097670 2024-09-13T17:41:44.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.692+0000 7f63fffff640 1 -- 172.21.15.102:0/1667929008 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f640801ca80 con 0x7f6418097670 2024-09-13T17:41:44.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.692+0000 7f641e500640 1 -- 172.21.15.102:0/1667929008 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64180937f0 con 0x7f6418097670 2024-09-13T17:41:44.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.694+0000 7f63fffff640 1 -- 172.21.15.102:0/1667929008 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f6408021060 con 0x7f6418097670 2024-09-13T17:41:44.711 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-13T17:41:44.709+0000 7f0b25e13ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T17:41:44.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.751+0000 7f641e500640 1 -- 172.21.15.102:0/1667929008 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6418092250 con 0x7f6418097670 2024-09-13T17:41:44.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.752+0000 7f63fffff640 1 -- 172.21.15.102:0/1667929008 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+4108 (secure 0 0 0) 0x7f6408017070 con 0x7f6418097670 2024-09-13T17:41:44.754 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:41:44.754 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":6,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:41:44.529778+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:41:44.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.753+0000 7f641e500640 1 -- 172.21.15.102:0/1667929008 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6418097670 msgr2=0x7f64180906d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:44.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.753+0000 7f641e500640 1 --2- 172.21.15.102:0/1667929008 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6418097670 0x7f64180906d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f640800ea30 tx=0x7f64080091d0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.753+0000 7f641e500640 1 -- 172.21.15.102:0/1667929008 shutdown_connections 2024-09-13T17:41:44.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.753+0000 7f641e500640 1 --2- 172.21.15.102:0/1667929008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f641809c1e0 0x7f6418091150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.753+0000 7f641e500640 1 --2- 172.21.15.102:0/1667929008 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f641809b290 0x7f6418090c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.753+0000 7f641e500640 1 --2- 172.21.15.102:0/1667929008 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6418097670 0x7f64180906d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:44.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.753+0000 7f641e500640 1 -- 172.21.15.102:0/1667929008 >> 172.21.15.102:0/1667929008 conn(0x7f641809f8b0 msgr2=0x7f6418099800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:44.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.753+0000 7f641e500640 1 -- 172.21.15.102:0/1667929008 shutdown_connections 2024-09-13T17:41:44.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:44.753+0000 7f641e500640 1 -- 172.21.15.102:0/1667929008 wait complete. 2024-09-13T17:41:44.763 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-13T17:41:44.761+0000 7f0b25e13ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T17:41:44.765 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-13T17:41:44.765 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-13T17:41:45.111 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-13T17:41:45.109+0000 7f0b25e13ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-13T17:41:45.147 INFO:tasks.ceph.osd.0.smithi102.stderr:2024-09-13T17:41:45.145+0000 7ff825f21440 -1 Falling back to public interface 2024-09-13T17:41:45.165 INFO:tasks.ceph.osd.1.smithi102.stderr:2024-09-13T17:41:45.163+0000 7fd0ce20e440 -1 Falling back to public interface 2024-09-13T17:41:45.168 DEBUG:teuthology.orchestra.run.smithi102:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T17:41:45.172 INFO:tasks.ceph.osd.2.smithi102.stderr:2024-09-13T17:41:45.170+0000 7f5359f23440 -1 Falling back to public interface 2024-09-13T17:41:45.177 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-13T17:41:45.175+0000 7f9a71412440 -1 Falling back to public interface 2024-09-13T17:41:45.194 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-13T17:41:45.193+0000 7f0b25e13ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T17:41:45.255 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.252+0000 7fd9f1e78640 1 Processor -- start 2024-09-13T17:41:45.255 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.253+0000 7fd9f1e78640 1 -- start start 2024-09-13T17:41:45.255 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.253+0000 7fd9f1e78640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd9ec132990 0x7fd9ec134d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:45.255 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.253+0000 7fd9f1e78640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd9ec1352b0 0x7fd9ec137690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:45.256 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.253+0000 7fd9f1e78640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9ec09c540 con 0x7fd9ec069ab0 2024-09-13T17:41:45.256 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.253+0000 7fd9f1e78640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9ec069e80 con 0x7fd9ec1352b0 2024-09-13T17:41:45.256 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.253+0000 7fd9f1e78640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec09cd20 con 0x7fd9ec132990 2024-09-13T17:41:45.256 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.253+0000 7fd9eaffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd9ec132990 0x7fd9ec134d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:45.256 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.253+0000 7fd9eb7fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fd9ec069ab0 0x7fd9ec12fde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:41922/0 (socket says 172.21.15.102:41922) 2024-09-13T17:41:45.256 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.253+0000 7fd9eb7fe640 1 -- 172.21.15.102:0/3392352848 learned_addr learned my addr 172.21.15.102:0/3392352848 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:41:45.256 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.253+0000 7fd9ebfff640 1 --2- 172.21.15.102:0/3392352848 >> v2:172.21.15.102:3300/0 conn(0x7fd9ec1352b0 0x7fd9ec137690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:45.256 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.253+0000 7fd9eaffd640 1 -- 172.21.15.102:0/3392352848 >> v2:172.21.15.102:3300/0 conn(0x7fd9ec1352b0 msgr2=0x7fd9ec137690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:41:45.256 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.253+0000 7fd9eaffd640 1 --2- 172.21.15.102:0/3392352848 >> v2:172.21.15.102:3300/0 conn(0x7fd9ec1352b0 0x7fd9ec137690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:45.256 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.253+0000 7fd9eaffd640 1 -- 172.21.15.102:0/3392352848 >> v1:172.21.15.102:6789/0 conn(0x7fd9ec069ab0 legacy=0x7fd9ec12fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:45.256 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.253+0000 7fd9eaffd640 1 -- 172.21.15.102:0/3392352848 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ec1304f0 con 0x7fd9ec132990 2024-09-13T17:41:45.259 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.254+0000 7fd9eaffd640 1 --2- 172.21.15.102:0/3392352848 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd9ec132990 0x7fd9ec134d70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc009850 tx=0x7fd9dc0090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65d0e8d4558979f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:45.259 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.254+0000 7fd9ea7fc640 1 -- 172.21.15.102:0/3392352848 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc02ac80 con 0x7fd9ec132990 2024-09-13T17:41:45.259 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.254+0000 7fd9ea7fc640 1 -- 172.21.15.102:0/3392352848 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9dc02ade0 con 0x7fd9ec132990 2024-09-13T17:41:45.260 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.254+0000 7fd9ea7fc640 1 -- 172.21.15.102:0/3392352848 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc032670 con 0x7fd9ec132990 2024-09-13T17:41:45.260 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.255+0000 7fd9f1e78640 1 -- 172.21.15.102:0/3392352848 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd9ec132990 msgr2=0x7fd9ec134d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:45.260 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.255+0000 7fd9f1e78640 1 --2- 172.21.15.102:0/3392352848 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd9ec132990 0x7fd9ec134d70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc009850 tx=0x7fd9dc0090f0 comp rx=0 tx=0).stop 2024-09-13T17:41:45.260 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.255+0000 7fd9f1e78640 1 -- 172.21.15.102:0/3392352848 shutdown_connections 2024-09-13T17:41:45.260 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.255+0000 7fd9f1e78640 1 --2- 172.21.15.102:0/3392352848 >> v2:172.21.15.102:3300/0 conn(0x7fd9ec1352b0 0x7fd9ec137690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:45.260 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.255+0000 7fd9f1e78640 1 --2- 172.21.15.102:0/3392352848 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd9ec132990 0x7fd9ec134d70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:45.260 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.255+0000 7fd9f1e78640 1 -- 172.21.15.102:0/3392352848 >> 172.21.15.102:0/3392352848 conn(0x7fd9ec070a00 msgr2=0x7fd9ec0a0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:45.260 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.255+0000 7fd9f1e78640 1 -- 172.21.15.102:0/3392352848 shutdown_connections 2024-09-13T17:41:45.260 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.255+0000 7fd9f1e78640 1 -- 172.21.15.102:0/3392352848 wait complete. 2024-09-13T17:41:45.260 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.256+0000 7fd9f1e78640 1 Processor -- start 2024-09-13T17:41:45.260 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.256+0000 7fd9f1e78640 1 -- start start 2024-09-13T17:41:45.260 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.256+0000 7fd9f1e78640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9ec069ab0 0x7fd9ec116e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:45.260 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.256+0000 7fd9f1e78640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd9ec132990 0x7fd9ec1173a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:45.261 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.256+0000 7fd9f1e78640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9ec1352b0 0x7fd9ec1178e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:45.261 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.256+0000 7fd9f1e78640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec119c20 con 0x7fd9ec069ab0 2024-09-13T17:41:45.261 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.256+0000 7fd9f1e78640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec0a07c0 con 0x7fd9ec1352b0 2024-09-13T17:41:45.261 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.256+0000 7fd9f1e78640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec068b80 con 0x7fd9ec132990 2024-09-13T17:41:45.261 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.256+0000 7fd9eb7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9ec069ab0 0x7fd9ec116e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:45.261 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.256+0000 7fd9ebfff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9ec1352b0 0x7fd9ec1178e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:45.261 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.256+0000 7fd9eaffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd9ec132990 0x7fd9ec1173a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:45.261 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.256+0000 7fd9eb7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9ec069ab0 0x7fd9ec116e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34286/0 (socket says 172.21.15.102:34286) 2024-09-13T17:41:45.261 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.256+0000 7fd9eb7fe640 1 -- 172.21.15.102:0/956623678 learned_addr learned my addr 172.21.15.102:0/956623678 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:41:45.261 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.257+0000 7fd9eaffd640 1 -- 172.21.15.102:0/956623678 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9ec1352b0 msgr2=0x7fd9ec1178e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:45.261 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.257+0000 7fd9eaffd640 1 --2- 172.21.15.102:0/956623678 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9ec1352b0 0x7fd9ec1178e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:45.261 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.257+0000 7fd9eaffd640 1 -- 172.21.15.102:0/956623678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9ec069ab0 msgr2=0x7fd9ec116e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:45.261 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.257+0000 7fd9eaffd640 1 --2- 172.21.15.102:0/956623678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9ec069ab0 0x7fd9ec116e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:45.261 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.257+0000 7fd9eaffd640 1 -- 172.21.15.102:0/956623678 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9dc009390 con 0x7fd9ec132990 2024-09-13T17:41:45.262 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.257+0000 7fd9ebfff640 1 --2- 172.21.15.102:0/956623678 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9ec1352b0 0x7fd9ec1178e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:41:45.262 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.257+0000 7fd9eaffd640 1 --2- 172.21.15.102:0/956623678 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd9ec132990 0x7fd9ec1173a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc031000 tx=0x7fd9dc032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:45.262 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.257+0000 7fd9d37fe640 1 -- 172.21.15.102:0/956623678 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc03d700 con 0x7fd9ec132990 2024-09-13T17:41:45.262 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.257+0000 7fd9d37fe640 1 -- 172.21.15.102:0/956623678 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9dc03d880 con 0x7fd9ec132990 2024-09-13T17:41:45.262 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.257+0000 7fd9f1e78640 1 -- 172.21.15.102:0/956623678 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9ec119e30 con 0x7fd9ec132990 2024-09-13T17:41:45.262 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.257+0000 7fd9d37fe640 1 -- 172.21.15.102:0/956623678 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc03dc40 con 0x7fd9ec132990 2024-09-13T17:41:45.262 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.257+0000 7fd9f1e78640 1 -- 172.21.15.102:0/956623678 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9ec06e6c0 con 0x7fd9ec132990 2024-09-13T17:41:45.262 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.257+0000 7fd9d37fe640 1 -- 172.21.15.102:0/956623678 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fd9dc039040 con 0x7fd9ec132990 2024-09-13T17:41:45.262 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.257+0000 7fd9d37fe640 1 -- 172.21.15.102:0/956623678 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7fd9dc0318b0 con 0x7fd9ec132990 2024-09-13T17:41:45.263 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.258+0000 7fd9f1e78640 1 -- 172.21.15.102:0/956623678 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9ec0714b0 con 0x7fd9ec132990 2024-09-13T17:41:45.263 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.260+0000 7fd9d37fe640 1 -- 172.21.15.102:0/956623678 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fd9dc037070 con 0x7fd9ec132990 2024-09-13T17:41:45.273 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-13T17:41:45.271+0000 7f0b25e13ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:41:45.319 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.317+0000 7fd9f1e78640 1 -- 172.21.15.102:0/956623678 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9ec06ec70 con 0x7fd9ec132990 2024-09-13T17:41:45.319 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.318+0000 7fd9d37fe640 1 -- 172.21.15.102:0/956623678 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+4108 (secure 0 0 0) 0x7fd9dc03b980 con 0x7fd9ec132990 2024-09-13T17:41:45.320 INFO:teuthology.misc.health.smithi102.stdout: 2024-09-13T17:41:45.320 INFO:teuthology.misc.health.smithi102.stdout:{"epoch":6,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:41:44.529778+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:41:45.320 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-13T17:41:45.318+0000 7f0b25e13ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T17:41:45.321 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.318+0000 7fd9f1e78640 1 -- 172.21.15.102:0/956623678 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd9ec132990 msgr2=0x7fd9ec1173a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:45.321 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.318+0000 7fd9f1e78640 1 --2- 172.21.15.102:0/956623678 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd9ec132990 0x7fd9ec1173a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc031000 tx=0x7fd9dc032000 comp rx=0 tx=0).stop 2024-09-13T17:41:45.322 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.318+0000 7fd9f1e78640 1 -- 172.21.15.102:0/956623678 shutdown_connections 2024-09-13T17:41:45.322 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.318+0000 7fd9f1e78640 1 --2- 172.21.15.102:0/956623678 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9ec1352b0 0x7fd9ec1178e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:45.322 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.318+0000 7fd9f1e78640 1 --2- 172.21.15.102:0/956623678 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd9ec132990 0x7fd9ec1173a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:45.322 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.318+0000 7fd9f1e78640 1 --2- 172.21.15.102:0/956623678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9ec069ab0 0x7fd9ec116e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:45.322 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.318+0000 7fd9f1e78640 1 -- 172.21.15.102:0/956623678 >> 172.21.15.102:0/956623678 conn(0x7fd9ec070a00 msgr2=0x7fd9ec09b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:45.322 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.319+0000 7fd9f1e78640 1 -- 172.21.15.102:0/956623678 shutdown_connections 2024-09-13T17:41:45.322 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:45.319+0000 7fd9f1e78640 1 -- 172.21.15.102:0/956623678 wait complete. 2024-09-13T17:41:45.329 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-09-13T17:41:45.402 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-13T17:41:45.400+0000 7f0b25e13ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T17:41:45.454 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-13T17:41:45.453+0000 7f0b25e13ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T17:41:45.497 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-13T17:41:45.496+0000 7f0b25e13ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T17:41:45.575 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-13T17:41:45.573+0000 7f0b25e13ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T17:41:45.623 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-13T17:41:45.621+0000 7f0b25e13ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T17:41:45.674 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-13T17:41:45.672+0000 7f0b25e13ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T17:41:45.741 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-13T17:41:45.739+0000 7f0b25e13ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T17:41:45.784 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-13T17:41:45.782+0000 7f0b25e13ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T17:41:45.825 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-13T17:41:45.823+0000 7f0b25e13ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T17:41:46.407 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-13T17:41:46.405+0000 7f0b25e13ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T17:41:47.417 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-13T17:41:47.415+0000 7f0b0c74f640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-13T17:41:47.417 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-13T17:41:47.416+0000 7f0b0c74f640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-13T17:41:48.604 INFO:tasks.ceph.osd.0.smithi102.stderr:2024-09-13T17:41:48.602+0000 7ff825f21440 -1 osd.0 0 log_to_monitors true 2024-09-13T17:41:48.660 INFO:tasks.ceph.osd.2.smithi102.stderr:2024-09-13T17:41:48.658+0000 7f5359f23440 -1 osd.2 0 log_to_monitors true 2024-09-13T17:41:48.665 INFO:tasks.ceph.osd.1.smithi102.stderr:2024-09-13T17:41:48.664+0000 7fd0ce20e440 -1 osd.1 0 log_to_monitors true 2024-09-13T17:41:48.709 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-13T17:41:48.705+0000 7f9a71412440 -1 osd.3 0 log_to_monitors true 2024-09-13T17:41:51.733 DEBUG:teuthology.orchestra.run.smithi102:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T17:41:51.831 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.829+0000 7fa725f1d640 1 Processor -- start 2024-09-13T17:41:51.831 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.829+0000 7fa725f1d640 1 -- start start 2024-09-13T17:41:51.831 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.829+0000 7fa725f1d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa72006b860 0x7fa720131460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:51.831 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.829+0000 7fa725f1d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa7201319a0 0x7fa720133d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:51.831 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.829+0000 7fa725f1d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa72006eb30 con 0x7fa72006e470 2024-09-13T17:41:51.832 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.829+0000 7fa725f1d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa72006e840 con 0x7fa7201319a0 2024-09-13T17:41:51.832 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.829+0000 7fa725f1d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa72006dfc0 con 0x7fa72006b860 2024-09-13T17:41:51.832 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.829+0000 7fa71f7fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fa72006e470 0x7fa72006b150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:41958/0 (socket says 172.21.15.102:41958) 2024-09-13T17:41:51.832 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.829+0000 7fa71ffff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa7201319a0 0x7fa720133d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:51.832 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.829+0000 7fa71f7fe640 1 -- 172.21.15.102:0/2412168450 learned_addr learned my addr 172.21.15.102:0/2412168450 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:41:51.832 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.829+0000 7fa71effd640 1 --2- 172.21.15.102:0/2412168450 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa72006b860 0x7fa720131460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:51.833 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.830+0000 7fa71ffff640 1 -- 172.21.15.102:0/2412168450 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa72006b860 msgr2=0x7fa720131460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:41:51.833 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.830+0000 7fa71e7fc640 1 -- 172.21.15.102:0/2412168450 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1880119862 0 0) 0x7fa72006eb30 con 0x7fa72006e470 2024-09-13T17:41:51.834 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.830+0000 7fa71ffff640 1 --2- 172.21.15.102:0/2412168450 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa72006b860 0x7fa720131460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:51.834 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.830+0000 7fa71ffff640 1 -- 172.21.15.102:0/2412168450 >> v1:172.21.15.102:6789/0 conn(0x7fa72006e470 legacy=0x7fa72006b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:51.834 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.830+0000 7fa71ffff640 1 -- 172.21.15.102:0/2412168450 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72006d9b0 con 0x7fa7201319a0 2024-09-13T17:41:51.834 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.830+0000 7fa71effd640 1 --2- 172.21.15.102:0/2412168450 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa72006b860 0x7fa720131460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:41:51.834 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.831+0000 7fa71ffff640 1 --2- 172.21.15.102:0/2412168450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7201319a0 0x7fa720133d80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa718009e90 tx=0x7fa718009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d95daf19c5f485 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:51.834 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.831+0000 7fa71e7fc640 1 -- 172.21.15.102:0/2412168450 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71802aea0 con 0x7fa7201319a0 2024-09-13T17:41:51.834 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.831+0000 7fa71e7fc640 1 -- 172.21.15.102:0/2412168450 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa71802d050 con 0x7fa7201319a0 2024-09-13T17:41:51.834 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.831+0000 7fa71e7fc640 1 -- 172.21.15.102:0/2412168450 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa718007780 con 0x7fa7201319a0 2024-09-13T17:41:51.834 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.831+0000 7fa725f1d640 1 -- 172.21.15.102:0/2412168450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7201319a0 msgr2=0x7fa720133d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:51.834 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.831+0000 7fa725f1d640 1 --2- 172.21.15.102:0/2412168450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7201319a0 0x7fa720133d80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa718009e90 tx=0x7fa718009780 comp rx=0 tx=0).stop 2024-09-13T17:41:51.834 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.831+0000 7fa725f1d640 1 -- 172.21.15.102:0/2412168450 shutdown_connections 2024-09-13T17:41:51.834 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.831+0000 7fa725f1d640 1 --2- 172.21.15.102:0/2412168450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7201319a0 0x7fa720133d80 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:51.834 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.831+0000 7fa725f1d640 1 --2- 172.21.15.102:0/2412168450 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa72006b860 0x7fa720131460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:51.835 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.831+0000 7fa725f1d640 1 -- 172.21.15.102:0/2412168450 >> 172.21.15.102:0/2412168450 conn(0x7fa7200709b0 msgr2=0x7fa72009f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:51.835 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.831+0000 7fa725f1d640 1 -- 172.21.15.102:0/2412168450 shutdown_connections 2024-09-13T17:41:51.835 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.831+0000 7fa725f1d640 1 -- 172.21.15.102:0/2412168450 wait complete. 2024-09-13T17:41:51.837 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.833+0000 7fa725f1d640 1 Processor -- start 2024-09-13T17:41:51.837 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.833+0000 7fa725f1d640 1 -- start start 2024-09-13T17:41:51.838 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.833+0000 7fa725f1d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa72006b860 0x7fa72011b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:51.838 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.833+0000 7fa725f1d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa72006e470 0x7fa72011b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:51.838 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.833+0000 7fa725f1d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7201319a0 0x7fa72011bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:51.838 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.833+0000 7fa725f1d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa720134930 con 0x7fa7201319a0 2024-09-13T17:41:51.838 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.833+0000 7fa725f1d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa72006d140 con 0x7fa72006e470 2024-09-13T17:41:51.838 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.833+0000 7fa725f1d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa72011e330 con 0x7fa72006b860 2024-09-13T17:41:51.838 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.833+0000 7fa71effd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa72006e470 0x7fa72011b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:51.838 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.833+0000 7fa71f7fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa72006b860 0x7fa72011b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:51.838 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.833+0000 7fa71effd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa72006e470 0x7fa72011b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:43314/0 (socket says 172.21.15.102:43314) 2024-09-13T17:41:51.838 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.833+0000 7fa71effd640 1 -- 172.21.15.102:0/3353739479 learned_addr learned my addr 172.21.15.102:0/3353739479 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:41:51.838 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.833+0000 7fa71ffff640 1 --2- 172.21.15.102:0/3353739479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7201319a0 0x7fa72011bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:51.838 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.834+0000 7fa71effd640 1 -- 172.21.15.102:0/3353739479 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa72006b860 msgr2=0x7fa72011b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:51.838 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.834+0000 7fa71effd640 1 --2- 172.21.15.102:0/3353739479 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa72006b860 0x7fa72011b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:51.839 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.834+0000 7fa71effd640 1 -- 172.21.15.102:0/3353739479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7201319a0 msgr2=0x7fa72011bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:51.839 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.834+0000 7fa71effd640 1 --2- 172.21.15.102:0/3353739479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7201319a0 0x7fa72011bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:51.839 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.834+0000 7fa71effd640 1 -- 172.21.15.102:0/3353739479 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa714009390 con 0x7fa72006e470 2024-09-13T17:41:51.839 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.834+0000 7fa71f7fe640 1 --2- 172.21.15.102:0/3353739479 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa72006b860 0x7fa72011b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:41:51.839 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.834+0000 7fa71ffff640 1 --2- 172.21.15.102:0/3353739479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7201319a0 0x7fa72011bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:41:51.839 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.834+0000 7fa71effd640 1 --2- 172.21.15.102:0/3353739479 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa72006e470 0x7fa72011b7e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa71400eaa0 tx=0x7fa7140090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:51.839 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.834+0000 7fa7077fe640 1 -- 172.21.15.102:0/3353739479 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71400ca10 con 0x7fa72006e470 2024-09-13T17:41:51.839 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.834+0000 7fa7077fe640 1 -- 172.21.15.102:0/3353739479 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa71400cb70 con 0x7fa72006e470 2024-09-13T17:41:51.839 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.834+0000 7fa725f1d640 1 -- 172.21.15.102:0/3353739479 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa718009a20 con 0x7fa72006e470 2024-09-13T17:41:51.839 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.834+0000 7fa7077fe640 1 -- 172.21.15.102:0/3353739479 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa714013470 con 0x7fa72006e470 2024-09-13T17:41:51.839 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.834+0000 7fa725f1d640 1 -- 172.21.15.102:0/3353739479 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa72011cfd0 con 0x7fa72006e470 2024-09-13T17:41:51.839 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.836+0000 7fa7077fe640 1 -- 172.21.15.102:0/3353739479 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 4) v1 ==== 36590+0+0 (secure 0 0 0) 0x7fa7140137e0 con 0x7fa72006e470 2024-09-13T17:41:51.839 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.836+0000 7fa725f1d640 1 -- 172.21.15.102:0/3353739479 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7200714b0 con 0x7fa72006e470 2024-09-13T17:41:51.839 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.836+0000 7fa7077fe640 1 -- 172.21.15.102:0/3353739479 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1509+0+0 (secure 0 0 0) 0x7fa714041e60 con 0x7fa72006e470 2024-09-13T17:41:51.844 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:51.842+0000 7fa7077fe640 1 -- 172.21.15.102:0/3353739479 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fa71401ab40 con 0x7fa72006e470 2024-09-13T17:41:52.113 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:52.111+0000 7fa725f1d640 1 -- 172.21.15.102:0/3353739479 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa720068760 con 0x7fa72006e470 2024-09-13T17:41:52.114 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:52.112+0000 7fa7077fe640 1 -- 172.21.15.102:0/3353739479 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v8) v1 ==== 74+0+4108 (secure 0 0 0) 0x7fa71401a360 con 0x7fa72006e470 2024-09-13T17:41:52.114 INFO:teuthology.misc.health.smithi102.stdout: 2024-09-13T17:41:52.115 INFO:teuthology.misc.health.smithi102.stdout:{"epoch":8,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:41:50.416914+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:41:52.117 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:52.116+0000 7fa725f1d640 1 -- 172.21.15.102:0/3353739479 >> v1:172.21.15.102:6816/34937 conn(0x7fa70c0310d0 legacy=0x7fa70c033580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:52.117 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:52.116+0000 7fa725f1d640 1 -- 172.21.15.102:0/3353739479 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa72006e470 msgr2=0x7fa72011b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:52.118 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:52.116+0000 7fa725f1d640 1 --2- 172.21.15.102:0/3353739479 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa72006e470 0x7fa72011b7e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa71400eaa0 tx=0x7fa7140090f0 comp rx=0 tx=0).stop 2024-09-13T17:41:52.118 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:52.116+0000 7fa725f1d640 1 -- 172.21.15.102:0/3353739479 shutdown_connections 2024-09-13T17:41:52.118 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:52.116+0000 7fa725f1d640 1 --2- 172.21.15.102:0/3353739479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7201319a0 0x7fa72011bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:52.118 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:52.116+0000 7fa725f1d640 1 --2- 172.21.15.102:0/3353739479 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa72006e470 0x7fa72011b7e0 secure :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa71400eaa0 tx=0x7fa7140090f0 comp rx=0 tx=0).stop 2024-09-13T17:41:52.118 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:52.116+0000 7fa725f1d640 1 --2- 172.21.15.102:0/3353739479 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa72006b860 0x7fa72011b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:52.118 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:52.116+0000 7fa725f1d640 1 -- 172.21.15.102:0/3353739479 >> 172.21.15.102:0/3353739479 conn(0x7fa7200709b0 msgr2=0x7fa720133ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:52.118 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:52.117+0000 7fa725f1d640 1 -- 172.21.15.102:0/3353739479 shutdown_connections 2024-09-13T17:41:52.118 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:52.117+0000 7fa725f1d640 1 -- 172.21.15.102:0/3353739479 wait complete. 2024-09-13T17:41:52.131 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-09-13T17:41:52.530 INFO:tasks.ceph.osd.2.smithi102.stderr:2024-09-13T17:41:52.527+0000 7f53549f6640 -1 osd.2 0 waiting for initial osdmap 2024-09-13T17:41:52.538 INFO:tasks.ceph.osd.2.smithi102.stderr:2024-09-13T17:41:52.537+0000 7f534d9e8640 -1 osd.2 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:41:52.575 INFO:tasks.ceph.osd.0.smithi102.stderr:2024-09-13T17:41:52.573+0000 7ff8221f9640 -1 osd.0 0 waiting for initial osdmap 2024-09-13T17:41:52.579 INFO:tasks.ceph.osd.0.smithi102.stderr:2024-09-13T17:41:52.578+0000 7ff819884640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:41:52.594 INFO:tasks.ceph.osd.1.smithi102.stderr:2024-09-13T17:41:52.592+0000 7fd0c89f6640 -1 osd.1 0 waiting for initial osdmap 2024-09-13T17:41:52.599 INFO:tasks.ceph.osd.1.smithi102.stderr:2024-09-13T17:41:52.598+0000 7fd0c1884640 -1 osd.1 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:41:52.606 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-13T17:41:52.605+0000 7f9a6bbf6640 -1 osd.3 0 waiting for initial osdmap 2024-09-13T17:41:52.609 INFO:tasks.ceph.osd.3.smithi102.stderr:2024-09-13T17:41:52.608+0000 7f9a64be8640 -1 osd.3 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:41:58.535 DEBUG:teuthology.orchestra.run.smithi102:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T17:41:58.630 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.628+0000 7f31e0290640 1 Processor -- start 2024-09-13T17:41:58.630 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.628+0000 7f31e0290640 1 -- start start 2024-09-13T17:41:58.631 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.629+0000 7f31e0290640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f31d812f8e0 0x7f31d812fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:58.631 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.629+0000 7f31e0290640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f31d809a610 0x7f31d8137790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:58.631 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.629+0000 7f31e0290640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31d809b770 con 0x7f31d8099b30 2024-09-13T17:41:58.631 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.629+0000 7f31e0290640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31d812f440 con 0x7f31d809a610 2024-09-13T17:41:58.632 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.629+0000 7f31e0290640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31d812f6e0 con 0x7f31d812f8e0 2024-09-13T17:41:58.632 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.629+0000 7f31de82d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f31d809a610 0x7f31d8137790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:58.632 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.629+0000 7f31de02c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f31d812f8e0 0x7f31d812fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:58.632 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.629+0000 7f31dd82b640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f31d8099b30 0x7f31d8099f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:59658/0 (socket says 172.21.15.102:59658) 2024-09-13T17:41:58.632 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.629+0000 7f31de82d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f31d809a610 0x7f31d8137790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47978/0 (socket says 172.21.15.102:47978) 2024-09-13T17:41:58.632 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.629+0000 7f31dd82b640 1 -- 172.21.15.102:0/523887774 learned_addr learned my addr 172.21.15.102:0/523887774 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:41:58.632 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.629+0000 7f31de02c640 1 -- 172.21.15.102:0/523887774 >> v2:172.21.15.102:3300/0 conn(0x7f31d809a610 msgr2=0x7f31d8137790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:41:58.632 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.629+0000 7f31de02c640 1 --2- 172.21.15.102:0/523887774 >> v2:172.21.15.102:3300/0 conn(0x7f31d809a610 0x7f31d8137790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:58.632 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.629+0000 7f31de02c640 1 -- 172.21.15.102:0/523887774 >> v1:172.21.15.102:6789/0 conn(0x7f31d8099b30 legacy=0x7f31d8099f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:58.632 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.629+0000 7f31de02c640 1 -- 172.21.15.102:0/523887774 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d809b0b0 con 0x7f31d812f8e0 2024-09-13T17:41:58.632 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.629+0000 7f31dd02a640 1 -- 172.21.15.102:0/523887774 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2464138711 0 0) 0x7f31d809b770 con 0x7f31d8099b30 2024-09-13T17:41:58.632 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.629+0000 7f31de82d640 1 --2- 172.21.15.102:0/523887774 >> v2:172.21.15.102:3300/0 conn(0x7f31d809a610 0x7f31d8137790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:41:58.636 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.630+0000 7f31de02c640 1 --2- 172.21.15.102:0/523887774 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f31d812f8e0 0x7f31d812fcb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f31d4009850 tx=0x7f31d40090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebb8934396b9dff0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:58.636 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.630+0000 7f31dd02a640 1 -- 172.21.15.102:0/523887774 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d402ad00 con 0x7f31d812f8e0 2024-09-13T17:41:58.636 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.630+0000 7f31dd02a640 1 -- 172.21.15.102:0/523887774 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d402ae60 con 0x7f31d812f8e0 2024-09-13T17:41:58.637 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.630+0000 7f31dd02a640 1 -- 172.21.15.102:0/523887774 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d4032670 con 0x7f31d812f8e0 2024-09-13T17:41:58.637 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.631+0000 7f31e0290640 1 -- 172.21.15.102:0/523887774 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f31d812f8e0 msgr2=0x7f31d812fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:58.637 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.631+0000 7f31e0290640 1 --2- 172.21.15.102:0/523887774 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f31d812f8e0 0x7f31d812fcb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f31d4009850 tx=0x7f31d40090f0 comp rx=0 tx=0).stop 2024-09-13T17:41:58.637 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.631+0000 7f31e0290640 1 -- 172.21.15.102:0/523887774 shutdown_connections 2024-09-13T17:41:58.637 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.631+0000 7f31e0290640 1 --2- 172.21.15.102:0/523887774 >> v2:172.21.15.102:3300/0 conn(0x7f31d809a610 0x7f31d8137790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:58.637 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.631+0000 7f31e0290640 1 --2- 172.21.15.102:0/523887774 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f31d812f8e0 0x7f31d812fcb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:58.637 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.631+0000 7f31e0290640 1 -- 172.21.15.102:0/523887774 >> 172.21.15.102:0/523887774 conn(0x7f31d809f070 msgr2=0x7f31d809f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:58.637 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.631+0000 7f31e0290640 1 -- 172.21.15.102:0/523887774 shutdown_connections 2024-09-13T17:41:58.637 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.631+0000 7f31e0290640 1 -- 172.21.15.102:0/523887774 wait complete. 2024-09-13T17:41:58.637 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.632+0000 7f31e0290640 1 Processor -- start 2024-09-13T17:41:58.637 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.632+0000 7f31e0290640 1 -- start start 2024-09-13T17:41:58.637 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.632+0000 7f31e0290640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f31d8099b30 0x7f31d813ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:58.638 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.632+0000 7f31e0290640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f31d809a610 0x7f31d813f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:58.638 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.632+0000 7f31e0290640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31d812f8e0 0x7f31d813f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:58.638 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.632+0000 7f31e0290640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d8126c20 con 0x7f31d812f8e0 2024-09-13T17:41:58.638 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.632+0000 7f31e0290640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d809c250 con 0x7f31d8099b30 2024-09-13T17:41:58.638 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.632+0000 7f31e0290640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31d8129f60 con 0x7f31d809a610 2024-09-13T17:41:58.638 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.632+0000 7f31dd82b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f31d809a610 0x7f31d813f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:58.638 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.632+0000 7f31de82d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31d812f8e0 0x7f31d813f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:58.638 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.632+0000 7f31de02c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f31d8099b30 0x7f31d813ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:58.638 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.632+0000 7f31de82d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31d812f8e0 0x7f31d813f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47986/0 (socket says 172.21.15.102:47986) 2024-09-13T17:41:58.638 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.632+0000 7f31de82d640 1 -- 172.21.15.102:0/3555613237 learned_addr learned my addr 172.21.15.102:0/3555613237 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:41:58.638 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.633+0000 7f31dd82b640 1 -- 172.21.15.102:0/3555613237 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f31d8099b30 msgr2=0x7f31d813ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:58.638 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.633+0000 7f31dd82b640 1 --2- 172.21.15.102:0/3555613237 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f31d8099b30 0x7f31d813ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:58.638 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.633+0000 7f31dd82b640 1 -- 172.21.15.102:0/3555613237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31d812f8e0 msgr2=0x7f31d813f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:58.639 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.633+0000 7f31dd82b640 1 --2- 172.21.15.102:0/3555613237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31d812f8e0 0x7f31d813f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:58.639 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.633+0000 7f31dd82b640 1 -- 172.21.15.102:0/3555613237 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d0009a20 con 0x7f31d809a610 2024-09-13T17:41:58.639 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.633+0000 7f31de82d640 1 --2- 172.21.15.102:0/3555613237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31d812f8e0 0x7f31d813f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:41:58.639 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.633+0000 7f31dd82b640 1 --2- 172.21.15.102:0/3555613237 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f31d809a610 0x7f31d813f2c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f31c8000c00 tx=0x7f31c8007c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:58.639 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.633+0000 7f31ce7fc640 1 -- 172.21.15.102:0/3555613237 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c8005aa0 con 0x7f31d809a610 2024-09-13T17:41:58.639 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.633+0000 7f31ce7fc640 1 -- 172.21.15.102:0/3555613237 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c8005c40 con 0x7f31d809a610 2024-09-13T17:41:58.639 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.633+0000 7f31e0290640 1 -- 172.21.15.102:0/3555613237 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d4009390 con 0x7f31d809a610 2024-09-13T17:41:58.639 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.633+0000 7f31ce7fc640 1 -- 172.21.15.102:0/3555613237 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c80070d0 con 0x7f31d809a610 2024-09-13T17:41:58.639 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.633+0000 7f31e0290640 1 -- 172.21.15.102:0/3555613237 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d8128330 con 0x7f31d809a610 2024-09-13T17:41:58.639 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.634+0000 7f31ce7fc640 1 -- 172.21.15.102:0/3555613237 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f31c80074c0 con 0x7f31d809a610 2024-09-13T17:41:58.639 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.634+0000 7f31ce7fc640 1 -- 172.21.15.102:0/3555613237 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2455+0+0 (secure 0 0 0) 0x7f31c8040f10 con 0x7f31d809a610 2024-09-13T17:41:58.639 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.634+0000 7f31e0290640 1 -- 172.21.15.102:0/3555613237 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31d80714b0 con 0x7f31d809a610 2024-09-13T17:41:58.640 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.639+0000 7f31ce7fc640 1 -- 172.21.15.102:0/3555613237 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f31c8017020 con 0x7f31d809a610 2024-09-13T17:41:58.794 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.793+0000 7f31e0290640 1 -- 172.21.15.102:0/3555613237 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31d8098dc0 con 0x7f31d809a610 2024-09-13T17:41:58.795 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.793+0000 7f31ce7fc640 1 -- 172.21.15.102:0/3555613237 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+6575 (secure 0 0 0) 0x7f31c8024070 con 0x7f31d809a610 2024-09-13T17:41:58.795 INFO:teuthology.misc.health.smithi102.stdout: 2024-09-13T17:41:58.795 INFO:teuthology.misc.health.smithi102.stdout:{"epoch":12,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:41:56.535559+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:41:58.798 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.795+0000 7f31e0290640 1 -- 172.21.15.102:0/3555613237 >> v1:172.21.15.102:6816/34937 conn(0x7f31bc031160 legacy=0x7f31bc033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:58.798 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.795+0000 7f31e0290640 1 -- 172.21.15.102:0/3555613237 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f31d809a610 msgr2=0x7f31d813f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:58.798 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.795+0000 7f31e0290640 1 --2- 172.21.15.102:0/3555613237 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f31d809a610 0x7f31d813f2c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f31c8000c00 tx=0x7f31c8007c80 comp rx=0 tx=0).stop 2024-09-13T17:41:58.798 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.795+0000 7f31e0290640 1 -- 172.21.15.102:0/3555613237 shutdown_connections 2024-09-13T17:41:58.798 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.795+0000 7f31e0290640 1 --2- 172.21.15.102:0/3555613237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31d812f8e0 0x7f31d813f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:58.798 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.795+0000 7f31e0290640 1 --2- 172.21.15.102:0/3555613237 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f31d809a610 0x7f31d813f2c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:58.798 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.795+0000 7f31e0290640 1 --2- 172.21.15.102:0/3555613237 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f31d8099b30 0x7f31d813ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:58.798 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.795+0000 7f31e0290640 1 -- 172.21.15.102:0/3555613237 >> 172.21.15.102:0/3555613237 conn(0x7f31d809f070 msgr2=0x7f31d809dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:58.798 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.795+0000 7f31e0290640 1 -- 172.21.15.102:0/3555613237 shutdown_connections 2024-09-13T17:41:58.798 INFO:teuthology.misc.health.smithi102.stderr:2024-09-13T17:41:58.795+0000 7f31e0290640 1 -- 172.21.15.102:0/3555613237 wait complete. 2024-09-13T17:41:58.806 DEBUG:teuthology.misc:4 of 4 OSDs are up 2024-09-13T17:41:58.806 INFO:tasks.ceph:Creating RBD pool 2024-09-13T17:41:58.806 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-13T17:41:58.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.897+0000 7f26adc94640 1 Processor -- start 2024-09-13T17:41:58.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.897+0000 7f26adc94640 1 -- start start 2024-09-13T17:41:58.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.897+0000 7f26adc94640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f26a8096a70 0x7f26a80987c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:58.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.897+0000 7f26adc94640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f26a8099110 0x7f26a812c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:58.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.897+0000 7f26adc94640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26a80999e0 con 0x7f26a8098d00 2024-09-13T17:41:58.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.897+0000 7f26adc94640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26a8069c30 con 0x7f26a8099110 2024-09-13T17:41:58.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.897+0000 7f26adc94640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a806ed00 con 0x7f26a8096a70 2024-09-13T17:41:58.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.897+0000 7f26a77fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f26a8096a70 0x7f26a80987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:58.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.897+0000 7f26a7fff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f26a8099110 0x7f26a812c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:58.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.897+0000 7f26a77fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f26a8096a70 0x7f26a80987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:47926/0 (socket says 172.21.15.102:47926) 2024-09-13T17:41:58.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.897+0000 7f26a6ffd640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f26a8098d00 0x7f26a812c2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:59670/0 (socket says 172.21.15.102:59670) 2024-09-13T17:41:58.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.897+0000 7f26a7fff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f26a8099110 0x7f26a812c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47988/0 (socket says 172.21.15.102:47988) 2024-09-13T17:41:58.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.897+0000 7f26a77fe640 1 -- 172.21.15.102:0/2556828272 learned_addr learned my addr 172.21.15.102:0/2556828272 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:41:58.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.898+0000 7f26a77fe640 1 -- 172.21.15.102:0/2556828272 >> v2:172.21.15.102:3300/0 conn(0x7f26a8099110 msgr2=0x7f26a812c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:41:58.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.898+0000 7f26a77fe640 1 --2- 172.21.15.102:0/2556828272 >> v2:172.21.15.102:3300/0 conn(0x7f26a8099110 0x7f26a812c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:58.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.898+0000 7f26a77fe640 1 -- 172.21.15.102:0/2556828272 >> v1:172.21.15.102:6789/0 conn(0x7f26a8098d00 legacy=0x7f26a812c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:58.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.898+0000 7f26a77fe640 1 -- 172.21.15.102:0/2556828272 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a80997a0 con 0x7f26a8096a70 2024-09-13T17:41:58.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.898+0000 7f26a67fc640 1 -- 172.21.15.102:0/2556828272 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2999174304 0 0) 0x7f26a80999e0 con 0x7f26a8098d00 2024-09-13T17:41:58.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.898+0000 7f26a7fff640 1 --2- 172.21.15.102:0/2556828272 >> v2:172.21.15.102:3300/0 conn(0x7f26a8099110 0x7f26a812c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:41:58.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.898+0000 7f26a77fe640 1 --2- 172.21.15.102:0/2556828272 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f26a8096a70 0x7f26a80987c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f269800bbf0 tx=0x7f269800b460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf04790a813e2da9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:58.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.899+0000 7f26a67fc640 1 -- 172.21.15.102:0/2556828272 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2698032620 con 0x7f26a8096a70 2024-09-13T17:41:58.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.899+0000 7f26a67fc640 1 -- 172.21.15.102:0/2556828272 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f269802e040 con 0x7f26a8096a70 2024-09-13T17:41:58.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.899+0000 7f26a67fc640 1 -- 172.21.15.102:0/2556828272 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26980329b0 con 0x7f26a8096a70 2024-09-13T17:41:58.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.899+0000 7f26adc94640 1 -- 172.21.15.102:0/2556828272 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f26a8096a70 msgr2=0x7f26a80987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:58.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.899+0000 7f26adc94640 1 --2- 172.21.15.102:0/2556828272 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f26a8096a70 0x7f26a80987c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f269800bbf0 tx=0x7f269800b460 comp rx=0 tx=0).stop 2024-09-13T17:41:58.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.899+0000 7f26adc94640 1 -- 172.21.15.102:0/2556828272 shutdown_connections 2024-09-13T17:41:58.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.899+0000 7f26adc94640 1 --2- 172.21.15.102:0/2556828272 >> v2:172.21.15.102:3300/0 conn(0x7f26a8099110 0x7f26a812c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:58.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.899+0000 7f26adc94640 1 --2- 172.21.15.102:0/2556828272 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f26a8096a70 0x7f26a80987c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:58.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.899+0000 7f26adc94640 1 -- 172.21.15.102:0/2556828272 >> 172.21.15.102:0/2556828272 conn(0x7f26a809e720 msgr2=0x7f26a809eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:58.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.899+0000 7f26adc94640 1 -- 172.21.15.102:0/2556828272 shutdown_connections 2024-09-13T17:41:58.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.899+0000 7f26adc94640 1 -- 172.21.15.102:0/2556828272 wait complete. 2024-09-13T17:41:58.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.900+0000 7f26adc94640 1 Processor -- start 2024-09-13T17:41:58.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.900+0000 7f26adc94640 1 -- start start 2024-09-13T17:41:58.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.900+0000 7f26adc94640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f26a8096a70 0x7f26a811d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:58.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.900+0000 7f26adc94640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26a8098d00 0x7f26a8119bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:58.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.900+0000 7f26adc94640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f26a8099110 0x7f26a811a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:58.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.900+0000 7f26adc94640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a811cc50 con 0x7f26a8098d00 2024-09-13T17:41:58.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.900+0000 7f26adc94640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a811b0c0 con 0x7f26a8096a70 2024-09-13T17:41:58.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.900+0000 7f26adc94640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f26a811b810 con 0x7f26a8099110 2024-09-13T17:41:58.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.901+0000 7f26a77fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f26a8096a70 0x7f26a811d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:58.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.901+0000 7f26a77fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f26a8096a70 0x7f26a811d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:47934/0 (socket says 172.21.15.102:47934) 2024-09-13T17:41:58.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.901+0000 7f26a77fe640 1 -- 172.21.15.102:0/2088841883 learned_addr learned my addr 172.21.15.102:0/2088841883 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:41:58.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.901+0000 7f26a7fff640 1 --2- 172.21.15.102:0/2088841883 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f26a8099110 0x7f26a811a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:58.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.901+0000 7f26a6ffd640 1 --2- 172.21.15.102:0/2088841883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26a8098d00 0x7f26a8119bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:58.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.901+0000 7f26a77fe640 1 -- 172.21.15.102:0/2088841883 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f26a8099110 msgr2=0x7f26a811a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:58.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.901+0000 7f26a77fe640 1 --2- 172.21.15.102:0/2088841883 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f26a8099110 0x7f26a811a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:58.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.901+0000 7f26a77fe640 1 -- 172.21.15.102:0/2088841883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26a8098d00 msgr2=0x7f26a8119bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:58.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.901+0000 7f26a77fe640 1 --2- 172.21.15.102:0/2088841883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26a8098d00 0x7f26a8119bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:58.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.901+0000 7f26a77fe640 1 -- 172.21.15.102:0/2088841883 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a0009a20 con 0x7f26a8096a70 2024-09-13T17:41:58.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.901+0000 7f26a77fe640 1 --2- 172.21.15.102:0/2088841883 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f26a8096a70 0x7f26a811d430 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f269802cdf0 tx=0x7f269803a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:58.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.901+0000 7f268f7fe640 1 -- 172.21.15.102:0/2088841883 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2698000c40 con 0x7f26a8096a70 2024-09-13T17:41:58.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.901+0000 7f268f7fe640 1 -- 172.21.15.102:0/2088841883 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f269803ba80 con 0x7f26a8096a70 2024-09-13T17:41:58.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.901+0000 7f268f7fe640 1 -- 172.21.15.102:0/2088841883 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269803be40 con 0x7f26a8096a70 2024-09-13T17:41:58.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.901+0000 7f26adc94640 1 -- 172.21.15.102:0/2088841883 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f269800b700 con 0x7f26a8096a70 2024-09-13T17:41:58.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.901+0000 7f26adc94640 1 -- 172.21.15.102:0/2088841883 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a811a7f0 con 0x7f26a8096a70 2024-09-13T17:41:58.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.902+0000 7f268f7fe640 1 -- 172.21.15.102:0/2088841883 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f269803c430 con 0x7f26a8096a70 2024-09-13T17:41:58.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.902+0000 7f268f7fe640 1 -- 172.21.15.102:0/2088841883 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2455+0+0 (secure 0 0 0) 0x7f2698042030 con 0x7f26a8096a70 2024-09-13T17:41:58.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.903+0000 7f26adc94640 1 -- 172.21.15.102:0/2088841883 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26a811a9e0 con 0x7f26a8096a70 2024-09-13T17:41:58.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:58.907+0000 7f268f7fe640 1 -- 172.21.15.102:0/2088841883 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f269804c020 con 0x7f26a8096a70 2024-09-13T17:41:59.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.058+0000 7f26adc94640 1 -- 172.21.15.102:0/2088841883 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f26a8064000 con 0x7f26a8096a70 2024-09-13T17:41:59.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.111+0000 7f268f7fe640 1 -- 172.21.15.102:0/2088841883 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v13) v1 ==== 109+0+0 (secure 0 0 0) 0x7f2698036070 con 0x7f26a8096a70 2024-09-13T17:41:59.113 INFO:teuthology.orchestra.run.smithi102.stderr:pool 'rbd' created 2024-09-13T17:41:59.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.114+0000 7f26adc94640 1 -- 172.21.15.102:0/2088841883 >> v1:172.21.15.102:6816/34937 conn(0x7f2694031160 legacy=0x7f2694033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:59.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.114+0000 7f26adc94640 1 -- 172.21.15.102:0/2088841883 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f26a8096a70 msgr2=0x7f26a811d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:59.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.114+0000 7f26adc94640 1 --2- 172.21.15.102:0/2088841883 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f26a8096a70 0x7f26a811d430 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f269802cdf0 tx=0x7f269803a000 comp rx=0 tx=0).stop 2024-09-13T17:41:59.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.114+0000 7f26adc94640 1 -- 172.21.15.102:0/2088841883 shutdown_connections 2024-09-13T17:41:59.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.114+0000 7f26adc94640 1 --2- 172.21.15.102:0/2088841883 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f26a8099110 0x7f26a811a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:59.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.114+0000 7f26adc94640 1 --2- 172.21.15.102:0/2088841883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26a8098d00 0x7f26a8119bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:59.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.114+0000 7f26adc94640 1 --2- 172.21.15.102:0/2088841883 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f26a8096a70 0x7f26a811d430 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:59.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.114+0000 7f26adc94640 1 -- 172.21.15.102:0/2088841883 >> 172.21.15.102:0/2088841883 conn(0x7f26a809e720 msgr2=0x7f26a80971f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:59.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.114+0000 7f26adc94640 1 -- 172.21.15.102:0/2088841883 shutdown_connections 2024-09-13T17:41:59.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.114+0000 7f26adc94640 1 -- 172.21.15.102:0/2088841883 wait complete. 2024-09-13T17:41:59.126 DEBUG:teuthology.orchestra.run.smithi102:> rbd --cluster ceph pool init rbd 2024-09-13T17:41:59.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.163+0000 7f841fa37b80 1 Processor -- start 2024-09-13T17:41:59.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.164+0000 7f841fa37b80 1 -- start start 2024-09-13T17:41:59.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.164+0000 7f841fa37b80 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5607b7480160 0x5607b7479290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:59.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.164+0000 7f841fa37b80 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5607b750e880 0x5607b7478370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:59.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.164+0000 7f841fa37b80 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x5607b73a40e0 con 0x5607b750e4b0 2024-09-13T17:41:59.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.164+0000 7f841fa37b80 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x5607b73dd260 con 0x5607b750e880 2024-09-13T17:41:59.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.164+0000 7f841fa37b80 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x5607b73d8f40 con 0x5607b7480160 2024-09-13T17:41:59.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.164+0000 7f841c94c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5607b7480160 0x5607b7479290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:59.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.164+0000 7f841d94e640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5607b750e880 0x5607b7478370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:59.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.164+0000 7f841c94c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5607b7480160 0x5607b7479290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48200/0 (socket says 172.21.15.102:48200) 2024-09-13T17:41:59.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.164+0000 7f841d14d640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x5607b750e4b0 0x5607b75119c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:59672/0 (socket says 172.21.15.102:59672) 2024-09-13T17:41:59.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.164+0000 7f841d94e640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5607b750e880 0x5607b7478370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:47948/0 (socket says 172.21.15.102:47948) 2024-09-13T17:41:59.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.164+0000 7f841c94c640 1 -- 172.21.15.102:0/559482257 learned_addr learned my addr 172.21.15.102:0/559482257 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:41:59.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.164+0000 7f841d94e640 1 -- 172.21.15.102:0/559482257 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5607b7480160 msgr2=0x5607b7479290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:41:59.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.164+0000 7f841d94e640 1 --2- 172.21.15.102:0/559482257 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5607b7480160 0x5607b7479290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:59.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.164+0000 7f8417fff640 1 -- 172.21.15.102:0/559482257 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2539553295 0 0) 0x5607b73a40e0 con 0x5607b750e4b0 2024-09-13T17:41:59.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.164+0000 7f841d94e640 1 -- 172.21.15.102:0/559482257 >> v1:172.21.15.102:6789/0 conn(0x5607b750e4b0 legacy=0x5607b75119c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:59.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.164+0000 7f841d94e640 1 -- 172.21.15.102:0/559482257 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607b737d270 con 0x5607b750e880 2024-09-13T17:41:59.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.164+0000 7f841c94c640 1 --2- 172.21.15.102:0/559482257 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5607b7480160 0x5607b7479290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:41:59.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.166+0000 7f841d94e640 1 --2- 172.21.15.102:0/559482257 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5607b750e880 0x5607b7478370 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f841000c960 tx=0x7f8410049d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac5d1529f4bf5a22 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:59.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.166+0000 7f8417fff640 1 -- 172.21.15.102:0/559482257 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841000c1a0 con 0x5607b750e880 2024-09-13T17:41:59.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.166+0000 7f8417fff640 1 -- 172.21.15.102:0/559482257 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f841000c300 con 0x5607b750e880 2024-09-13T17:41:59.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.166+0000 7f8417fff640 1 -- 172.21.15.102:0/559482257 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8410051520 con 0x5607b750e880 2024-09-13T17:41:59.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.167+0000 7f841fa37b80 1 -- 172.21.15.102:0/559482257 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5607b750e880 msgr2=0x5607b7478370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:59.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.167+0000 7f841fa37b80 1 --2- 172.21.15.102:0/559482257 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5607b750e880 0x5607b7478370 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f841000c960 tx=0x7f8410049d10 comp rx=0 tx=0).stop 2024-09-13T17:41:59.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.167+0000 7f841fa37b80 1 -- 172.21.15.102:0/559482257 shutdown_connections 2024-09-13T17:41:59.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.167+0000 7f841fa37b80 1 --2- 172.21.15.102:0/559482257 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5607b750e880 0x5607b7478370 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:59.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.167+0000 7f841fa37b80 1 --2- 172.21.15.102:0/559482257 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5607b7480160 0x5607b7479290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:59.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.167+0000 7f841fa37b80 1 -- 172.21.15.102:0/559482257 >> 172.21.15.102:0/559482257 conn(0x5607b7481980 msgr2=0x5607b7481d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:59.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.167+0000 7f841fa37b80 1 -- 172.21.15.102:0/559482257 shutdown_connections 2024-09-13T17:41:59.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.167+0000 7f841fa37b80 1 -- 172.21.15.102:0/559482257 wait complete. 2024-09-13T17:41:59.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.168+0000 7f841fa37b80 1 Processor -- start 2024-09-13T17:41:59.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.168+0000 7f841fa37b80 1 -- start start 2024-09-13T17:41:59.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.168+0000 7f841fa37b80 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5607b7480160 0x5607b751ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:59.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.168+0000 7f841fa37b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607b750e4b0 0x5607b751f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:59.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.168+0000 7f841fa37b80 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5607b750e880 0x5607b74fb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:59.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.168+0000 7f841fa37b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607b739bba0 con 0x5607b750e4b0 2024-09-13T17:41:59.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.168+0000 7f841fa37b80 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x5607b739b840 con 0x5607b750e880 2024-09-13T17:41:59.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.168+0000 7f841fa37b80 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x5607b735e7a0 con 0x5607b7480160 2024-09-13T17:41:59.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.168+0000 7f841d94e640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5607b750e880 0x5607b74fb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:59.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.168+0000 7f841d14d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5607b7480160 0x5607b751ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:59.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.168+0000 7f841d94e640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5607b750e880 0x5607b74fb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:47954/0 (socket says 172.21.15.102:47954) 2024-09-13T17:41:59.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.168+0000 7f841c94c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607b750e4b0 0x5607b751f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:59.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.168+0000 7f841d94e640 1 -- 172.21.15.102:0/3760739594 learned_addr learned my addr 172.21.15.102:0/3760739594 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:41:59.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.169+0000 7f841d14d640 1 -- 172.21.15.102:0/3760739594 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5607b750e880 msgr2=0x5607b74fb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:59.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.169+0000 7f841d14d640 1 --2- 172.21.15.102:0/3760739594 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5607b750e880 0x5607b74fb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:59.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.169+0000 7f841d14d640 1 -- 172.21.15.102:0/3760739594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607b750e4b0 msgr2=0x5607b751f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:59.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.169+0000 7f841d14d640 1 --2- 172.21.15.102:0/3760739594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607b750e4b0 0x5607b751f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:59.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.169+0000 7f841d14d640 1 -- 172.21.15.102:0/3760739594 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8408005bc0 con 0x5607b7480160 2024-09-13T17:41:59.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.169+0000 7f841d94e640 1 --2- 172.21.15.102:0/3760739594 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5607b750e880 0x5607b74fb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:41:59.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.169+0000 7f841c94c640 1 --2- 172.21.15.102:0/3760739594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607b750e4b0 0x5607b751f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:41:59.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.169+0000 7f841d14d640 1 --2- 172.21.15.102:0/3760739594 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5607b7480160 0x5607b751ed50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f841800bed0 tx=0x7f84180017f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:59.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.169+0000 7f84157fa640 1 -- 172.21.15.102:0/3760739594 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841800f450 con 0x5607b7480160 2024-09-13T17:41:59.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.169+0000 7f84157fa640 1 -- 172.21.15.102:0/3760739594 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f841800f5d0 con 0x5607b7480160 2024-09-13T17:41:59.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.169+0000 7f841fa37b80 1 -- 172.21.15.102:0/3760739594 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f841000c4f0 con 0x5607b7480160 2024-09-13T17:41:59.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.169+0000 7f84157fa640 1 -- 172.21.15.102:0/3760739594 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841800f990 con 0x5607b7480160 2024-09-13T17:41:59.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.169+0000 7f841fa37b80 1 -- 172.21.15.102:0/3760739594 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5607b738a0e0 con 0x5607b7480160 2024-09-13T17:41:59.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.170+0000 7f84157fa640 1 -- 172.21.15.102:0/3760739594 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f84180183b0 con 0x5607b7480160 2024-09-13T17:41:59.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.170+0000 7f84157fa640 1 -- 172.21.15.102:0/3760739594 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2797+0+0 (secure 0 0 0) 0x7f8418008030 con 0x5607b7480160 2024-09-13T17:41:59.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:41:59.170+0000 7f841fa37b80 1 -- 172.21.15.102:0/3760739594 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool application enable","pool": "rbd","app": "rbd"} v 0) v1 -- 0x5607b737d600 con 0x5607b7480160 2024-09-13T17:42:00.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:00.111+0000 7f84157fa640 1 -- 172.21.15.102:0/3760739594 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}]=0 enabled application 'rbd' on pool 'rbd' v14) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8418014020 con 0x5607b7480160 2024-09-13T17:42:00.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:00.112+0000 7f841fa37b80 1 -- 172.21.15.102:0/3760739594 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x5607b739b630 con 0x5607b7480160 2024-09-13T17:42:00.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:00.118+0000 7f84157fa640 1 -- 172.21.15.102:0/3760739594 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8418016e80 con 0x5607b7480160 2024-09-13T17:42:00.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:00.118+0000 7f84177fe640 1 -- 172.21.15.102:0/3760739594 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=14}) v3 -- 0x7f83f8002a10 con 0x5607b7480160 2024-09-13T17:42:00.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:00.118+0000 7f84157fa640 1 -- 172.21.15.102:0/3760739594 <== mon.2 v2:172.21.15.102:3302/0 8 ==== osd_map(14..14 src has 1..14) v4 ==== 642+0+0 (secure 0 0 0) 0x7f841801f640 con 0x5607b7480160 2024-09-13T17:42:00.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:00.118+0000 7f841fa37b80 1 -- 172.21.15.102:0/3760739594 --> v1:172.21.15.102:6812/35352 -- osd_op(unknown.0.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14) v8 -- 0x5607b7529900 con 0x5607b7525e00 2024-09-13T17:42:00.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:00.124+0000 7f841d94e640 1 -- 172.21.15.102:0/3760739594 <== osd.3 v1:172.21.15.102:6812/35352 1 ==== osd_op_reply(1 rbd_trash [create] v14'1 uv1 ondisk = 0) v8 ==== 153+0+0 (unknown 1742540995 0 0) 0x7f84100518e0 con 0x5607b7525e00 2024-09-13T17:42:00.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:00.126+0000 7f841fa37b80 1 -- 172.21.15.102:0/3760739594 --> v1:172.21.15.102:6812/35352 -- osd_op(unknown.0.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=33b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e14) v8 -- 0x5607b7576830 con 0x5607b7525e00 2024-09-13T17:42:00.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:00.126+0000 7f841d94e640 1 -- 172.21.15.102:0/3760739594 <== osd.3 v1:172.21.15.102:6812/35352 2 ==== osd_op_reply(2 rbd_info [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (unknown 173568156 0 0) 0x7f84100518e0 con 0x5607b7525e00 2024-09-13T17:42:00.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:00.126+0000 7f841fa37b80 1 -- 172.21.15.102:0/3760739594 --> v1:172.21.15.102:6812/35352 -- osd_op(unknown.0.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e14) v8 -- 0x5607b7575920 con 0x5607b7525e00 2024-09-13T17:42:00.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:00.126+0000 7f841d94e640 1 -- 172.21.15.102:0/3760739594 <== osd.3 v1:172.21.15.102:6812/35352 3 ==== osd_op_reply(3 rbd_info [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (unknown 185109136 0 0) 0x7f84100518e0 con 0x5607b7525e00 2024-09-13T17:42:00.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:00.126+0000 7f84177fe640 1 -- 172.21.15.102:0/3760739594 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- pool_op(create unmanaged snap pool 2 tid 4 name v0) v4 -- 0x7f83f8003db0 con 0x5607b7480160 2024-09-13T17:42:01.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:01.124+0000 7f84157fa640 1 -- 172.21.15.102:0/3760739594 <== mon.2 v2:172.21.15.102:3302/0 9 ==== pool_op_reply(tid 4 (0) Success v15) v1 ==== 55+0+0 (secure 0 0 0) 0x7f841801f8a0 con 0x5607b7480160 2024-09-13T17:42:01.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:01.124+0000 7f84157fa640 1 -- 172.21.15.102:0/3760739594 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=15}) v3 -- 0x7f8400036580 con 0x5607b7480160 2024-09-13T17:42:01.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:01.125+0000 7f84157fa640 1 -- 172.21.15.102:0/3760739594 <== mon.2 v2:172.21.15.102:3302/0 10 ==== osd_map(15..15 src has 1..15) v4 ==== 618+0+0 (secure 0 0 0) 0x7f841800fd80 con 0x5607b7480160 2024-09-13T17:42:01.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:01.125+0000 7f84177fe640 1 -- 172.21.15.102:0/3760739594 --> v1:172.21.15.102:6812/35352 -- osd_op(unknown.0.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15) v8 -- 0x7f83f80056f0 con 0x5607b7525e00 2024-09-13T17:42:01.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:01.130+0000 7f841d94e640 1 -- 172.21.15.102:0/3760739594 <== osd.3 v1:172.21.15.102:6812/35352 4 ==== osd_op_reply(5 rbd_info [create,write 0~8] v15'1 uv1 ondisk = 0) v8 ==== 194+0+0 (unknown 913822303 0 0) 0x7f84100518e0 con 0x5607b7525e00 2024-09-13T17:42:01.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:01.130+0000 7f84177fe640 1 -- 172.21.15.102:0/3760739594 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- pool_op(delete unmanaged snap pool 2 tid 6 name v15) v4 -- 0x7f83f80051c0 con 0x5607b7480160 2024-09-13T17:42:02.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.133+0000 7f84157fa640 1 -- 172.21.15.102:0/3760739594 <== mon.2 v2:172.21.15.102:3302/0 11 ==== pool_op_reply(tid 6 (0) Success v16) v1 ==== 43+0+0 (secure 0 0 0) 0x7f84180162c0 con 0x5607b7480160 2024-09-13T17:42:02.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.134+0000 7f84157fa640 1 -- 172.21.15.102:0/3760739594 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=16}) v3 -- 0x7f8400036710 con 0x5607b7480160 2024-09-13T17:42:02.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.134+0000 7f84157fa640 1 -- 172.21.15.102:0/3760739594 <== mon.2 v2:172.21.15.102:3302/0 12 ==== osd_map(16..16 src has 1..16) v4 ==== 646+0+0 (secure 0 0 0) 0x7f841801fad0 con 0x5607b7480160 2024-09-13T17:42:02.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.134+0000 7f84177fe640 1 -- 172.21.15.102:0/3760739594 --> v1:172.21.15.102:6812/35352 -- osd_op(unknown.0.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19 in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16) v8 -- 0x7f83f8005f90 con 0x5607b7525e00 2024-09-13T17:42:02.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.142+0000 7f841d94e640 1 -- 172.21.15.102:0/3760739594 <== osd.3 v1:172.21.15.102:6812/35352 5 ==== osd_op_reply(7 rbd_info [write 0~19] v16'2 uv2 ondisk = 0) v8 ==== 152+0+0 (unknown 2130366450 0 0) 0x7f84100518e0 con 0x5607b7525e00 2024-09-13T17:42:02.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.142+0000 7f841fa37b80 1 -- 172.21.15.102:0/3760739594 >> v1:172.21.15.102:6812/35352 conn(0x5607b7525e00 legacy=0x5607b75281e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:02.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.142+0000 7f841fa37b80 1 -- 172.21.15.102:0/3760739594 >> v1:172.21.15.102:6816/34937 conn(0x7f840002fa90 legacy=0x7f8400031f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:02.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.142+0000 7f841fa37b80 1 -- 172.21.15.102:0/3760739594 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5607b7480160 msgr2=0x5607b751ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:02.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.142+0000 7f841fa37b80 1 --2- 172.21.15.102:0/3760739594 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5607b7480160 0x5607b751ed50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f841800bed0 tx=0x7f84180017f0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.142+0000 7f841d94e640 1 -- 172.21.15.102:0/3760739594 reap_dead start 2024-09-13T17:42:02.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.142+0000 7f841fa37b80 1 -- 172.21.15.102:0/3760739594 shutdown_connections 2024-09-13T17:42:02.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.142+0000 7f841fa37b80 1 -- 172.21.15.102:0/3760739594 >> 172.21.15.102:0/3760739594 conn(0x5607b7481980 msgr2=0x5607b7481f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:02.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.142+0000 7f841fa37b80 1 -- 172.21.15.102:0/3760739594 shutdown_connections 2024-09-13T17:42:02.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.142+0000 7f841fa37b80 1 -- 172.21.15.102:0/3760739594 wait complete. 2024-09-13T17:42:02.148 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-13T17:42:02.149 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T17:42:02.149 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T17:42:02.191 INFO:tasks.ceph.mds.a:Started 2024-09-13T17:42:02.192 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-13T17:42:02.192 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:02.270 INFO:tasks.ceph.mds.a.smithi102.stdout:starting mds.a at 2024-09-13T17:42:02.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.296+0000 7f105b51d640 1 Processor -- start 2024-09-13T17:42:02.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.296+0000 7f105b51d640 1 -- start start 2024-09-13T17:42:02.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.297+0000 7f105b51d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1054099570 0x7f1054134720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:02.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.297+0000 7f105b51d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1054134c60 0x7f1054137040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:02.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.297+0000 7f105b51d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f105409a7c0 con 0x7f105409fff0 2024-09-13T17:42:02.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.297+0000 7f105b51d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10540a03c0 con 0x7f1054134c60 2024-09-13T17:42:02.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.297+0000 7f105b51d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f105409abb0 con 0x7f1054099570 2024-09-13T17:42:02.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.297+0000 7f1058ab8640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1054099570 0x7f1054134720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:02.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.297+0000 7f1059aba640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1054134c60 0x7f1054137040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:02.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.297+0000 7f10592b9640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f105409fff0 0x7f1054098e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33494/0 (socket says 172.21.15.102:33494) 2024-09-13T17:42:02.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.297+0000 7f1058ab8640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1054099570 0x7f1054134720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:54706/0 (socket says 172.21.15.102:54706) 2024-09-13T17:42:02.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.297+0000 7f1059aba640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1054134c60 0x7f1054137040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52306/0 (socket says 172.21.15.102:52306) 2024-09-13T17:42:02.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.297+0000 7f10592b9640 1 -- 172.21.15.102:0/3731729141 learned_addr learned my addr 172.21.15.102:0/3731729141 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:42:02.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.297+0000 7f1059aba640 1 -- 172.21.15.102:0/3731729141 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1054099570 msgr2=0x7f1054134720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:02.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.297+0000 7f1059aba640 1 --2- 172.21.15.102:0/3731729141 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1054099570 0x7f1054134720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.297+0000 7f1059aba640 1 -- 172.21.15.102:0/3731729141 >> v1:172.21.15.102:6789/0 conn(0x7f105409fff0 legacy=0x7f1054098e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:02.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.297+0000 7f1059aba640 1 -- 172.21.15.102:0/3731729141 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1054099940 con 0x7f1054134c60 2024-09-13T17:42:02.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.297+0000 7f104bfff640 1 -- 172.21.15.102:0/3731729141 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2234998571 0 0) 0x7f105409a7c0 con 0x7f105409fff0 2024-09-13T17:42:02.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.298+0000 7f1059aba640 1 --2- 172.21.15.102:0/3731729141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1054134c60 0x7f1054137040 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f104c009e60 tx=0x7f104c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c8e2d4c625fa72 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:02.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.298+0000 7f104bfff640 1 -- 172.21.15.102:0/3731729141 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c007550 con 0x7f1054134c60 2024-09-13T17:42:02.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.298+0000 7f104bfff640 1 -- 172.21.15.102:0/3731729141 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f104c02d050 con 0x7f1054134c60 2024-09-13T17:42:02.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.298+0000 7f104bfff640 1 -- 172.21.15.102:0/3731729141 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c0078e0 con 0x7f1054134c60 2024-09-13T17:42:02.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.298+0000 7f105b51d640 1 -- 172.21.15.102:0/3731729141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1054134c60 msgr2=0x7f1054137040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:02.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.298+0000 7f105b51d640 1 --2- 172.21.15.102:0/3731729141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1054134c60 0x7f1054137040 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f104c009e60 tx=0x7f104c009780 comp rx=0 tx=0).stop 2024-09-13T17:42:02.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.299+0000 7f105b51d640 1 -- 172.21.15.102:0/3731729141 shutdown_connections 2024-09-13T17:42:02.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.299+0000 7f105b51d640 1 --2- 172.21.15.102:0/3731729141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1054134c60 0x7f1054137040 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.299+0000 7f105b51d640 1 --2- 172.21.15.102:0/3731729141 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1054099570 0x7f1054134720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.299+0000 7f105b51d640 1 -- 172.21.15.102:0/3731729141 >> 172.21.15.102:0/3731729141 conn(0x7f105405e160 msgr2=0x7f105409f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:02.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.299+0000 7f105b51d640 1 -- 172.21.15.102:0/3731729141 shutdown_connections 2024-09-13T17:42:02.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.299+0000 7f105b51d640 1 -- 172.21.15.102:0/3731729141 wait complete. 2024-09-13T17:42:02.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.299+0000 7f105b51d640 1 Processor -- start 2024-09-13T17:42:02.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.299+0000 7f105b51d640 1 -- start start 2024-09-13T17:42:02.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.300+0000 7f105b51d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1054099570 0x7f10541220d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:02.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.300+0000 7f105b51d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f105409fff0 0x7f105411e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:02.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.300+0000 7f105b51d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1054134c60 0x7f105411e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:02.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.300+0000 7f105b51d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105409b510 con 0x7f105409fff0 2024-09-13T17:42:02.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.300+0000 7f105b51d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1054121630 con 0x7f1054134c60 2024-09-13T17:42:02.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.300+0000 7f105b51d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1054121bd0 con 0x7f1054099570 2024-09-13T17:42:02.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.300+0000 7f10592b9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1054099570 0x7f10541220d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:02.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.300+0000 7f1058ab8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f105409fff0 0x7f105411e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:02.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.300+0000 7f1058ab8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f105409fff0 0x7f105411e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52308/0 (socket says 172.21.15.102:52308) 2024-09-13T17:42:02.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.300+0000 7f1059aba640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1054134c60 0x7f105411e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:02.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.300+0000 7f1058ab8640 1 -- 172.21.15.102:0/850978300 learned_addr learned my addr 172.21.15.102:0/850978300 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:02.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.300+0000 7f1058ab8640 1 -- 172.21.15.102:0/850978300 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1054099570 msgr2=0x7f10541220d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:02.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.300+0000 7f1058ab8640 1 --2- 172.21.15.102:0/850978300 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1054099570 0x7f10541220d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.300+0000 7f1058ab8640 1 -- 172.21.15.102:0/850978300 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1054134c60 msgr2=0x7f105411e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:02.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.300+0000 7f1058ab8640 1 --2- 172.21.15.102:0/850978300 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1054134c60 0x7f105411e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.300+0000 7f1058ab8640 1 -- 172.21.15.102:0/850978300 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f104c009a20 con 0x7f105409fff0 2024-09-13T17:42:02.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.301+0000 7f1058ab8640 1 --2- 172.21.15.102:0/850978300 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f105409fff0 0x7f105411e1b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f104400ea20 tx=0x7f10440090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:02.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.301+0000 7f10497fa640 1 -- 172.21.15.102:0/850978300 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104400ca10 con 0x7f105409fff0 2024-09-13T17:42:02.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.301+0000 7f10497fa640 1 -- 172.21.15.102:0/850978300 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f104400cb70 con 0x7f105409fff0 2024-09-13T17:42:02.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.301+0000 7f10497fa640 1 -- 172.21.15.102:0/850978300 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1044013470 con 0x7f105409fff0 2024-09-13T17:42:02.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.301+0000 7f105b51d640 1 -- 172.21.15.102:0/850978300 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1054134a50 con 0x7f105409fff0 2024-09-13T17:42:02.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.301+0000 7f105b51d640 1 -- 172.21.15.102:0/850978300 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105411f3f0 con 0x7f105409fff0 2024-09-13T17:42:02.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.302+0000 7f10497fa640 1 -- 172.21.15.102:0/850978300 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f1044013860 con 0x7f105409fff0 2024-09-13T17:42:02.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.302+0000 7f105b51d640 1 -- 172.21.15.102:0/850978300 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f105411f6a0 con 0x7f105409fff0 2024-09-13T17:42:02.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.302+0000 7f10497fa640 1 -- 172.21.15.102:0/850978300 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f1044042040 con 0x7f105409fff0 2024-09-13T17:42:02.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.307+0000 7f10497fa640 1 -- 172.21.15.102:0/850978300 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f1044021020 con 0x7f105409fff0 2024-09-13T17:42:02.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.461+0000 7f105b51d640 1 -- 172.21.15.102:0/850978300 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10541341f0 con 0x7f105409fff0 2024-09-13T17:42:02.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.461+0000 7f10497fa640 1 -- 172.21.15.102:0/850978300 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+8201 (secure 0 0 0) 0x7f10440232f0 con 0x7f105409fff0 2024-09-13T17:42:02.464 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:42:02.464 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":16,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:42:02.123403+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:02.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.463+0000 7f105b51d640 1 -- 172.21.15.102:0/850978300 >> v1:172.21.15.102:6816/34937 conn(0x7f1038031160 legacy=0x7f1038033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:02.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.463+0000 7f105b51d640 1 -- 172.21.15.102:0/850978300 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f105409fff0 msgr2=0x7f105411e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:02.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.463+0000 7f105b51d640 1 --2- 172.21.15.102:0/850978300 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f105409fff0 0x7f105411e1b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f104400ea20 tx=0x7f10440090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.463+0000 7f105b51d640 1 -- 172.21.15.102:0/850978300 shutdown_connections 2024-09-13T17:42:02.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.463+0000 7f105b51d640 1 --2- 172.21.15.102:0/850978300 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1054134c60 0x7f105411e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.463+0000 7f105b51d640 1 --2- 172.21.15.102:0/850978300 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f105409fff0 0x7f105411e1b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.463+0000 7f105b51d640 1 --2- 172.21.15.102:0/850978300 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1054099570 0x7f10541220d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.463+0000 7f105b51d640 1 -- 172.21.15.102:0/850978300 >> 172.21.15.102:0/850978300 conn(0x7f105405e160 msgr2=0x7f105406e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:02.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.463+0000 7f105b51d640 1 -- 172.21.15.102:0/850978300 shutdown_connections 2024-09-13T17:42:02.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.463+0000 7f105b51d640 1 -- 172.21.15.102:0/850978300 wait complete. 2024-09-13T17:42:02.475 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:54.442612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T17:41:59.076782+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 16, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2024-09-13T17:42:02.476 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:42:02.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.569+0000 7fd552b09640 1 Processor -- start 2024-09-13T17:42:02.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.569+0000 7fd552b09640 1 -- start start 2024-09-13T17:42:02.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.570+0000 7fd552b09640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd54c06d670 0x7fd54c06e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:02.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.570+0000 7fd552b09640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd54c09f7f0 0x7fd54c12d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:02.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.570+0000 7fd552b09640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd54c12d7c0 0x7fd54c12dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:02.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.570+0000 7fd552b09640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd54c06da40 con 0x7fd54c06d670 2024-09-13T17:42:02.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.570+0000 7fd552b09640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c09fbc0 con 0x7fd54c12d7c0 2024-09-13T17:42:02.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.570+0000 7fd552b09640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c06e210 con 0x7fd54c09f7f0 2024-09-13T17:42:02.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.570+0000 7fd5508a5640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd54c06d670 0x7fd54c06e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:02.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.570+0000 7fd54bfff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd54c09f7f0 0x7fd54c12d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:02.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.570+0000 7fd5510a6640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd54c12d7c0 0x7fd54c12dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:02.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.570+0000 7fd5508a5640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd54c06d670 0x7fd54c06e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52316/0 (socket says 172.21.15.102:52316) 2024-09-13T17:42:02.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.570+0000 7fd5508a5640 1 -- 172.21.15.102:0/569375726 learned_addr learned my addr 172.21.15.102:0/569375726 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:02.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.570+0000 7fd5508a5640 1 -- 172.21.15.102:0/569375726 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd54c09f7f0 msgr2=0x7fd54c12d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:02.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.570+0000 7fd5508a5640 1 --2- 172.21.15.102:0/569375726 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd54c09f7f0 0x7fd54c12d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.570+0000 7fd5508a5640 1 -- 172.21.15.102:0/569375726 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd54c12d7c0 msgr2=0x7fd54c12dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:02.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.570+0000 7fd5508a5640 1 --2- 172.21.15.102:0/569375726 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd54c12d7c0 0x7fd54c12dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.570+0000 7fd5508a5640 1 -- 172.21.15.102:0/569375726 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd54c09ff20 con 0x7fd54c06d670 2024-09-13T17:42:02.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.570+0000 7fd5510a6640 1 --2- 172.21.15.102:0/569375726 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd54c12d7c0 0x7fd54c12dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:02.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.570+0000 7fd54bfff640 1 --2- 172.21.15.102:0/569375726 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd54c09f7f0 0x7fd54c12d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:02.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.571+0000 7fd5508a5640 1 --2- 172.21.15.102:0/569375726 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd54c06d670 0x7fd54c06e780 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd53c009910 tx=0x7fd53c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5181c5b3f7af9c3c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:02.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.571+0000 7fd54b7fe640 1 -- 172.21.15.102:0/569375726 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c02ac80 con 0x7fd54c06d670 2024-09-13T17:42:02.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.571+0000 7fd54b7fe640 1 -- 172.21.15.102:0/569375726 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53c02ade0 con 0x7fd54c06d670 2024-09-13T17:42:02.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.571+0000 7fd54b7fe640 1 -- 172.21.15.102:0/569375726 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c032670 con 0x7fd54c06d670 2024-09-13T17:42:02.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.572+0000 7fd552b09640 1 -- 172.21.15.102:0/569375726 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd54c06d670 msgr2=0x7fd54c06e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:02.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.572+0000 7fd552b09640 1 --2- 172.21.15.102:0/569375726 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd54c06d670 0x7fd54c06e780 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd53c009910 tx=0x7fd53c0091d0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.572+0000 7fd552b09640 1 -- 172.21.15.102:0/569375726 shutdown_connections 2024-09-13T17:42:02.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.572+0000 7fd552b09640 1 --2- 172.21.15.102:0/569375726 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd54c12d7c0 0x7fd54c12dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.572+0000 7fd552b09640 1 --2- 172.21.15.102:0/569375726 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd54c09f7f0 0x7fd54c12d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.572+0000 7fd552b09640 1 --2- 172.21.15.102:0/569375726 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd54c06d670 0x7fd54c06e780 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.572+0000 7fd552b09640 1 -- 172.21.15.102:0/569375726 >> 172.21.15.102:0/569375726 conn(0x7fd54c067bb0 msgr2=0x7fd54c0a08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:02.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.572+0000 7fd552b09640 1 -- 172.21.15.102:0/569375726 shutdown_connections 2024-09-13T17:42:02.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.572+0000 7fd552b09640 1 -- 172.21.15.102:0/569375726 wait complete. 2024-09-13T17:42:02.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.572+0000 7fd552b09640 1 Processor -- start 2024-09-13T17:42:02.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.573+0000 7fd552b09640 1 -- start start 2024-09-13T17:42:02.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.573+0000 7fd552b09640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd54c06d670 0x7fd54c0682c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:02.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.573+0000 7fd552b09640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd54c09f7f0 0x7fd54c068800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:02.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.573+0000 7fd552b09640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd54c12d7c0 0x7fd54c068d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:02.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.573+0000 7fd552b09640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c11f0d0 con 0x7fd54c06d670 2024-09-13T17:42:02.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.573+0000 7fd552b09640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c09b470 con 0x7fd54c09f7f0 2024-09-13T17:42:02.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.573+0000 7fd552b09640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c06c200 con 0x7fd54c12d7c0 2024-09-13T17:42:02.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.573+0000 7fd5508a5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd54c06d670 0x7fd54c0682c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:02.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.573+0000 7fd5510a6640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd54c12d7c0 0x7fd54c068d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:02.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.573+0000 7fd54bfff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd54c09f7f0 0x7fd54c068800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:02.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.573+0000 7fd5508a5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd54c06d670 0x7fd54c0682c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52332/0 (socket says 172.21.15.102:52332) 2024-09-13T17:42:02.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.573+0000 7fd5510a6640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd54c12d7c0 0x7fd54c068d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:41828/0 (socket says 172.21.15.102:41828) 2024-09-13T17:42:02.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.573+0000 7fd5508a5640 1 -- 172.21.15.102:0/774444943 learned_addr learned my addr 172.21.15.102:0/774444943 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:02.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.573+0000 7fd54bfff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd54c09f7f0 0x7fd54c068800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:54724/0 (socket says 172.21.15.102:54724) 2024-09-13T17:42:02.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.573+0000 7fd54bfff640 1 -- 172.21.15.102:0/774444943 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd54c12d7c0 msgr2=0x7fd54c068d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:02.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.573+0000 7fd54bfff640 1 --2- 172.21.15.102:0/774444943 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd54c12d7c0 0x7fd54c068d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.574+0000 7fd54bfff640 1 -- 172.21.15.102:0/774444943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd54c06d670 msgr2=0x7fd54c0682c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:02.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.574+0000 7fd54bfff640 1 --2- 172.21.15.102:0/774444943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd54c06d670 0x7fd54c0682c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.574+0000 7fd54bfff640 1 -- 172.21.15.102:0/774444943 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd540009390 con 0x7fd54c09f7f0 2024-09-13T17:42:02.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.574+0000 7fd5508a5640 1 --2- 172.21.15.102:0/774444943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd54c06d670 0x7fd54c0682c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:02.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.574+0000 7fd54bfff640 1 --2- 172.21.15.102:0/774444943 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd54c09f7f0 0x7fd54c068800 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd54000ea50 tx=0x7fd5400090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:02.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.574+0000 7fd548ff9640 1 -- 172.21.15.102:0/774444943 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54000c940 con 0x7fd54c09f7f0 2024-09-13T17:42:02.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.574+0000 7fd548ff9640 1 -- 172.21.15.102:0/774444943 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd54000caa0 con 0x7fd54c09f7f0 2024-09-13T17:42:02.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.574+0000 7fd552b09640 1 -- 172.21.15.102:0/774444943 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd544009a20 con 0x7fd54c09f7f0 2024-09-13T17:42:02.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.574+0000 7fd548ff9640 1 -- 172.21.15.102:0/774444943 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54000ce40 con 0x7fd54c09f7f0 2024-09-13T17:42:02.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.574+0000 7fd552b09640 1 -- 172.21.15.102:0/774444943 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd53c009450 con 0x7fd54c09f7f0 2024-09-13T17:42:02.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.575+0000 7fd548ff9640 1 -- 172.21.15.102:0/774444943 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fd540013680 con 0x7fd54c09f7f0 2024-09-13T17:42:02.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.575+0000 7fd548ff9640 1 -- 172.21.15.102:0/774444943 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7fd540042100 con 0x7fd54c09f7f0 2024-09-13T17:42:02.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.575+0000 7fd552b09640 1 -- 172.21.15.102:0/774444943 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd54c064600 con 0x7fd54c09f7f0 2024-09-13T17:42:02.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.579+0000 7fd548ff9640 1 -- 172.21.15.102:0/774444943 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fd540023ad0 con 0x7fd54c09f7f0 2024-09-13T17:42:02.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.734+0000 7fd552b09640 1 -- 172.21.15.102:0/774444943 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd54c09af10 con 0x7fd54c09f7f0 2024-09-13T17:42:02.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.734+0000 7fd548ff9640 1 -- 172.21.15.102:0/774444943 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd5400232f0 con 0x7fd54c09f7f0 2024-09-13T17:42:02.736 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-09-13T17:42:02.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.736+0000 7fd552b09640 1 -- 172.21.15.102:0/774444943 >> v1:172.21.15.102:6816/34937 conn(0x7fd538031160 legacy=0x7fd538033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:02.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.736+0000 7fd552b09640 1 -- 172.21.15.102:0/774444943 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd54c09f7f0 msgr2=0x7fd54c068800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:02.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.736+0000 7fd552b09640 1 --2- 172.21.15.102:0/774444943 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd54c09f7f0 0x7fd54c068800 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd54000ea50 tx=0x7fd5400090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.736+0000 7fd552b09640 1 -- 172.21.15.102:0/774444943 shutdown_connections 2024-09-13T17:42:02.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.736+0000 7fd552b09640 1 --2- 172.21.15.102:0/774444943 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd54c12d7c0 0x7fd54c068d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.736+0000 7fd552b09640 1 --2- 172.21.15.102:0/774444943 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd54c09f7f0 0x7fd54c068800 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.736+0000 7fd552b09640 1 --2- 172.21.15.102:0/774444943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd54c06d670 0x7fd54c0682c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.736+0000 7fd552b09640 1 -- 172.21.15.102:0/774444943 >> 172.21.15.102:0/774444943 conn(0x7fd54c067bb0 msgr2=0x7fd54c09d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:02.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.736+0000 7fd552b09640 1 -- 172.21.15.102:0/774444943 shutdown_connections 2024-09-13T17:42:02.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.736+0000 7fd552b09640 1 -- 172.21.15.102:0/774444943 wait complete. 2024-09-13T17:42:02.748 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-13T17:42:02.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.841+0000 7f70e96c9640 1 Processor -- start 2024-09-13T17:42:02.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.841+0000 7f70e96c9640 1 -- start start 2024-09-13T17:42:02.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.842+0000 7f70e96c9640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f70e40689d0 0x7f70e409d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:02.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.842+0000 7f70e96c9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f70e409dd70 0x7f70e412f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:02.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.842+0000 7f70e96c9640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70e4096b10 con 0x7f70e409e160 2024-09-13T17:42:02.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.842+0000 7f70e96c9640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70e4096db0 con 0x7f70e40689d0 2024-09-13T17:42:02.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.842+0000 7f70e96c9640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f70e4097730 con 0x7f70e409dd70 2024-09-13T17:42:02.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.842+0000 7f70e27fc640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f70e409dd70 0x7f70e412f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:02.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.842+0000 7f70e27fc640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f70e409dd70 0x7f70e412f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:41840/0 (socket says 172.21.15.102:41840) 2024-09-13T17:42:02.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.842+0000 7f70e27fc640 1 -- 172.21.15.102:0/1576528722 learned_addr learned my addr 172.21.15.102:0/1576528722 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:02.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.842+0000 7f70e2ffd640 1 --2- 172.21.15.102:0/1576528722 >> v2:172.21.15.102:3300/0 conn(0x7f70e40689d0 0x7f70e409d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:02.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.842+0000 7f70e27fc640 1 -- 172.21.15.102:0/1576528722 >> v2:172.21.15.102:3300/0 conn(0x7f70e40689d0 msgr2=0x7f70e409d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:02.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.842+0000 7f70e27fc640 1 --2- 172.21.15.102:0/1576528722 >> v2:172.21.15.102:3300/0 conn(0x7f70e40689d0 0x7f70e409d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.842+0000 7f70e27fc640 1 -- 172.21.15.102:0/1576528722 >> v1:172.21.15.102:6789/0 conn(0x7f70e409e160 legacy=0x7f70e412fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:02.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.842+0000 7f70e27fc640 1 -- 172.21.15.102:0/1576528722 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e40696e0 con 0x7f70e409dd70 2024-09-13T17:42:02.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.843+0000 7f70e27fc640 1 --2- 172.21.15.102:0/1576528722 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f70e409dd70 0x7f70e412f610 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f70d8009910 tx=0x7f70d80091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbf23f616a78985f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:02.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.843+0000 7f70e1ffb640 1 -- 172.21.15.102:0/1576528722 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d802ad20 con 0x7f70e409dd70 2024-09-13T17:42:02.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.843+0000 7f70e1ffb640 1 -- 172.21.15.102:0/1576528722 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d802ae80 con 0x7f70e409dd70 2024-09-13T17:42:02.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.843+0000 7f70e1ffb640 1 -- 172.21.15.102:0/1576528722 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d8032670 con 0x7f70e409dd70 2024-09-13T17:42:02.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.844+0000 7f70e96c9640 1 -- 172.21.15.102:0/1576528722 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f70e409dd70 msgr2=0x7f70e412f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:02.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.844+0000 7f70e96c9640 1 --2- 172.21.15.102:0/1576528722 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f70e409dd70 0x7f70e412f610 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f70d8009910 tx=0x7f70d80091d0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.844+0000 7f70e96c9640 1 -- 172.21.15.102:0/1576528722 shutdown_connections 2024-09-13T17:42:02.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.844+0000 7f70e96c9640 1 --2- 172.21.15.102:0/1576528722 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f70e409dd70 0x7f70e412f610 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.844+0000 7f70e96c9640 1 --2- 172.21.15.102:0/1576528722 >> v2:172.21.15.102:3300/0 conn(0x7f70e40689d0 0x7f70e409d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.844+0000 7f70e96c9640 1 -- 172.21.15.102:0/1576528722 >> 172.21.15.102:0/1576528722 conn(0x7f70e406f320 msgr2=0x7f70e40a2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:02.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.844+0000 7f70e96c9640 1 -- 172.21.15.102:0/1576528722 shutdown_connections 2024-09-13T17:42:02.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.844+0000 7f70e96c9640 1 -- 172.21.15.102:0/1576528722 wait complete. 2024-09-13T17:42:02.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.845+0000 7f70e96c9640 1 Processor -- start 2024-09-13T17:42:02.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.845+0000 7f70e96c9640 1 -- start start 2024-09-13T17:42:02.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.845+0000 7f70e96c9640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f70e40689d0 0x7f70e411dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:02.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.845+0000 7f70e96c9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f70e409dd70 0x7f70e411e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:02.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.845+0000 7f70e96c9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70e409e160 0x7f70e411e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:02.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.845+0000 7f70e96c9640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e41211e0 con 0x7f70e409e160 2024-09-13T17:42:02.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.845+0000 7f70e96c9640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e4121780 con 0x7f70e40689d0 2024-09-13T17:42:02.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.845+0000 7f70e96c9640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f70e4097ee0 con 0x7f70e409dd70 2024-09-13T17:42:02.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.845+0000 7f70e27fc640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f70e409dd70 0x7f70e411e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:02.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.845+0000 7f70e2ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f70e40689d0 0x7f70e411dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:02.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.845+0000 7f70e37fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70e409e160 0x7f70e411e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:02.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.845+0000 7f70e27fc640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f70e409dd70 0x7f70e411e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:41842/0 (socket says 172.21.15.102:41842) 2024-09-13T17:42:02.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.845+0000 7f70e27fc640 1 -- 172.21.15.102:0/129789869 learned_addr learned my addr 172.21.15.102:0/129789869 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:02.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.845+0000 7f70e2ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f70e40689d0 0x7f70e411dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:54726/0 (socket says 172.21.15.102:54726) 2024-09-13T17:42:02.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.846+0000 7f70e37fe640 1 -- 172.21.15.102:0/129789869 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f70e409dd70 msgr2=0x7f70e411e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:02.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.846+0000 7f70e37fe640 1 --2- 172.21.15.102:0/129789869 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f70e409dd70 0x7f70e411e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.846+0000 7f70e37fe640 1 -- 172.21.15.102:0/129789869 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f70e40689d0 msgr2=0x7f70e411dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:02.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.846+0000 7f70e37fe640 1 --2- 172.21.15.102:0/129789869 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f70e40689d0 0x7f70e411dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.846+0000 7f70e37fe640 1 -- 172.21.15.102:0/129789869 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d8009450 con 0x7f70e409e160 2024-09-13T17:42:02.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.846+0000 7f70e27fc640 1 --2- 172.21.15.102:0/129789869 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f70e409dd70 0x7f70e411e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:02.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.846+0000 7f70e2ffd640 1 --2- 172.21.15.102:0/129789869 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f70e40689d0 0x7f70e411dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:02.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.846+0000 7f70e37fe640 1 --2- 172.21.15.102:0/129789869 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70e409e160 0x7f70e411e790 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f70e409e5a0 tx=0x7f70dc007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:02.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.846+0000 7f70caffd640 1 -- 172.21.15.102:0/129789869 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc011590 con 0x7f70e409e160 2024-09-13T17:42:02.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.846+0000 7f70caffd640 1 -- 172.21.15.102:0/129789869 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70dc0116f0 con 0x7f70e409e160 2024-09-13T17:42:02.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.846+0000 7f70e96c9640 1 -- 172.21.15.102:0/129789869 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e41200d0 con 0x7f70e409e160 2024-09-13T17:42:02.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.846+0000 7f70caffd640 1 -- 172.21.15.102:0/129789869 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc011ab0 con 0x7f70e409e160 2024-09-13T17:42:02.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.846+0000 7f70e96c9640 1 -- 172.21.15.102:0/129789869 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e411f8e0 con 0x7f70e409e160 2024-09-13T17:42:02.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.847+0000 7f70caffd640 1 -- 172.21.15.102:0/129789869 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f70dc01b3b0 con 0x7f70e409e160 2024-09-13T17:42:02.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.847+0000 7f70e96c9640 1 -- 172.21.15.102:0/129789869 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70e4121350 con 0x7f70e409e160 2024-09-13T17:42:02.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.847+0000 7f70caffd640 1 -- 172.21.15.102:0/129789869 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f70dc041110 con 0x7f70e409e160 2024-09-13T17:42:02.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.851+0000 7f70caffd640 1 -- 172.21.15.102:0/129789869 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f70dc01a360 con 0x7f70e409e160 2024-09-13T17:42:03.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:02.999+0000 7f70e96c9640 1 -- 172.21.15.102:0/129789869 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f70e4064000 con 0x7f70e409e160 2024-09-13T17:42:03.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.000+0000 7f70caffd640 1 -- 172.21.15.102:0/129789869 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7f70dc017070 con 0x7f70e409e160 2024-09-13T17:42:03.002 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-09-13T17:42:03.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.002+0000 7f70e96c9640 1 -- 172.21.15.102:0/129789869 >> v1:172.21.15.102:6816/34937 conn(0x7f70d0031160 legacy=0x7f70d0033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:03.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.002+0000 7f70e96c9640 1 -- 172.21.15.102:0/129789869 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70e409e160 msgr2=0x7f70e411e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:03.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.002+0000 7f70e96c9640 1 --2- 172.21.15.102:0/129789869 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70e409e160 0x7f70e411e790 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f70e409e5a0 tx=0x7f70dc007dd0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.002+0000 7f70e96c9640 1 -- 172.21.15.102:0/129789869 shutdown_connections 2024-09-13T17:42:03.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.002+0000 7f70e96c9640 1 --2- 172.21.15.102:0/129789869 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70e409e160 0x7f70e411e790 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.002+0000 7f70e96c9640 1 --2- 172.21.15.102:0/129789869 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f70e409dd70 0x7f70e411e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.002+0000 7f70e96c9640 1 --2- 172.21.15.102:0/129789869 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f70e40689d0 0x7f70e411dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.002+0000 7f70e96c9640 1 -- 172.21.15.102:0/129789869 >> 172.21.15.102:0/129789869 conn(0x7f70e406f320 msgr2=0x7f70e412f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:03.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.002+0000 7f70e96c9640 1 -- 172.21.15.102:0/129789869 shutdown_connections 2024-09-13T17:42:03.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.002+0000 7f70e96c9640 1 -- 172.21.15.102:0/129789869 wait complete. 2024-09-13T17:42:03.014 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:42:03.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.104+0000 7f392e280640 1 Processor -- start 2024-09-13T17:42:03.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.104+0000 7f392e280640 1 -- start start 2024-09-13T17:42:03.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.105+0000 7f392e280640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3928098ef0 0x7f39280992c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:03.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.105+0000 7f392e280640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3928067c30 0x7f3928137760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:03.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.105+0000 7f392e280640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f392809a0e0 con 0x7f3928067130 2024-09-13T17:42:03.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.105+0000 7f392e280640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3928097600 con 0x7f3928098ef0 2024-09-13T17:42:03.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.105+0000 7f392e280640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3928096ea0 con 0x7f3928067c30 2024-09-13T17:42:03.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.105+0000 7f3927fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3928098ef0 0x7f39280992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:03.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.105+0000 7f392c81d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3928067c30 0x7f3928137760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:03.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.105+0000 7f3927fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3928098ef0 0x7f39280992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:54730/0 (socket says 172.21.15.102:54730) 2024-09-13T17:42:03.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.105+0000 7f392c81d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3928067c30 0x7f3928137760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:41848/0 (socket says 172.21.15.102:41848) 2024-09-13T17:42:03.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.105+0000 7f3927fff640 1 -- 172.21.15.102:0/510784864 learned_addr learned my addr 172.21.15.102:0/510784864 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:03.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.105+0000 7f39277fe640 1 -- 172.21.15.102:0/510784864 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3428651116 0 0) 0x7f392809a0e0 con 0x7f3928067130 2024-09-13T17:42:03.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.105+0000 7f3927fff640 1 -- 172.21.15.102:0/510784864 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3928067c30 msgr2=0x7f3928137760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:03.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.105+0000 7f3927fff640 1 --2- 172.21.15.102:0/510784864 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3928067c30 0x7f3928137760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.105+0000 7f3927fff640 1 -- 172.21.15.102:0/510784864 >> v1:172.21.15.102:6789/0 conn(0x7f3928067130 legacy=0x7f3928067520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:03.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.105+0000 7f3927fff640 1 -- 172.21.15.102:0/510784864 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3928097cc0 con 0x7f3928098ef0 2024-09-13T17:42:03.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.105+0000 7f392c81d640 1 --2- 172.21.15.102:0/510784864 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3928067c30 0x7f3928137760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:03.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.106+0000 7f3927fff640 1 --2- 172.21.15.102:0/510784864 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3928098ef0 0x7f39280992c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3918009940 tx=0x7f39180091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29b47c1c69af23d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:03.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.106+0000 7f39277fe640 1 -- 172.21.15.102:0/510784864 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391802ad90 con 0x7f3928098ef0 2024-09-13T17:42:03.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.106+0000 7f39277fe640 1 -- 172.21.15.102:0/510784864 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3918032440 con 0x7f3928098ef0 2024-09-13T17:42:03.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.106+0000 7f39277fe640 1 -- 172.21.15.102:0/510784864 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39180327e0 con 0x7f3928098ef0 2024-09-13T17:42:03.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.107+0000 7f392e280640 1 -- 172.21.15.102:0/510784864 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3928098ef0 msgr2=0x7f39280992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:03.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.107+0000 7f392e280640 1 --2- 172.21.15.102:0/510784864 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3928098ef0 0x7f39280992c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3918009940 tx=0x7f39180091d0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.107+0000 7f392e280640 1 -- 172.21.15.102:0/510784864 shutdown_connections 2024-09-13T17:42:03.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.107+0000 7f392e280640 1 --2- 172.21.15.102:0/510784864 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3928067c30 0x7f3928137760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.107+0000 7f392e280640 1 --2- 172.21.15.102:0/510784864 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3928098ef0 0x7f39280992c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.107+0000 7f392e280640 1 -- 172.21.15.102:0/510784864 >> 172.21.15.102:0/510784864 conn(0x7f392805db50 msgr2=0x7f392805df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:03.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.107+0000 7f392e280640 1 -- 172.21.15.102:0/510784864 shutdown_connections 2024-09-13T17:42:03.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.107+0000 7f392e280640 1 -- 172.21.15.102:0/510784864 wait complete. 2024-09-13T17:42:03.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.108+0000 7f392e280640 1 Processor -- start 2024-09-13T17:42:03.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.108+0000 7f392e280640 1 -- start start 2024-09-13T17:42:03.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.108+0000 7f392e280640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3928067130 0x7f3928120940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:03.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.108+0000 7f392e280640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3928067c30 0x7f392811d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:03.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.108+0000 7f392e280640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3928098ef0 0x7f392811d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:03.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.108+0000 7f392e280640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392811fea0 con 0x7f3928067130 2024-09-13T17:42:03.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.108+0000 7f392e280640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f392811ea70 con 0x7f3928067c30 2024-09-13T17:42:03.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.108+0000 7f392e280640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f392809b460 con 0x7f3928098ef0 2024-09-13T17:42:03.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.108+0000 7f3927fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3928067130 0x7f3928120940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:03.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.108+0000 7f391ffff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3928067c30 0x7f392811d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:03.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.108+0000 7f3927fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3928067130 0x7f3928120940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52354/0 (socket says 172.21.15.102:52354) 2024-09-13T17:42:03.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.108+0000 7f3927fff640 1 -- 172.21.15.102:0/1720046915 learned_addr learned my addr 172.21.15.102:0/1720046915 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:03.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.108+0000 7f392c81d640 1 --2- 172.21.15.102:0/1720046915 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3928098ef0 0x7f392811d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:03.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.108+0000 7f391ffff640 1 -- 172.21.15.102:0/1720046915 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3928098ef0 msgr2=0x7f392811d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:03.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.108+0000 7f391ffff640 1 --2- 172.21.15.102:0/1720046915 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3928098ef0 0x7f392811d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.108+0000 7f391ffff640 1 -- 172.21.15.102:0/1720046915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3928067130 msgr2=0x7f3928120940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:03.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.108+0000 7f391ffff640 1 --2- 172.21.15.102:0/1720046915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3928067130 0x7f3928120940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.109+0000 7f391ffff640 1 -- 172.21.15.102:0/1720046915 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3920009a20 con 0x7f3928067c30 2024-09-13T17:42:03.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.109+0000 7f3927fff640 1 --2- 172.21.15.102:0/1720046915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3928067130 0x7f3928120940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:03.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.109+0000 7f392c81d640 1 --2- 172.21.15.102:0/1720046915 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3928098ef0 0x7f392811d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:03.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.109+0000 7f391ffff640 1 --2- 172.21.15.102:0/1720046915 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3928067c30 0x7f392811d310 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f392806a490 tx=0x7f3910007e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:03.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.109+0000 7f3924ff9640 1 -- 172.21.15.102:0/1720046915 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3910011590 con 0x7f3928067c30 2024-09-13T17:42:03.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.109+0000 7f3924ff9640 1 -- 172.21.15.102:0/1720046915 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3910011710 con 0x7f3928067c30 2024-09-13T17:42:03.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.109+0000 7f392e280640 1 -- 172.21.15.102:0/1720046915 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3918009450 con 0x7f3928067c30 2024-09-13T17:42:03.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.109+0000 7f3924ff9640 1 -- 172.21.15.102:0/1720046915 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3910011ad0 con 0x7f3928067c30 2024-09-13T17:42:03.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.109+0000 7f392e280640 1 -- 172.21.15.102:0/1720046915 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f392811dd90 con 0x7f3928067c30 2024-09-13T17:42:03.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.109+0000 7f3924ff9640 1 -- 172.21.15.102:0/1720046915 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f391001b3b0 con 0x7f3928067c30 2024-09-13T17:42:03.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.110+0000 7f3924ff9640 1 -- 172.21.15.102:0/1720046915 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f3910040fd0 con 0x7f3928067c30 2024-09-13T17:42:03.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.110+0000 7f392e280640 1 -- 172.21.15.102:0/1720046915 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3928064000 con 0x7f3928067c30 2024-09-13T17:42:03.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.114+0000 7f3924ff9640 1 -- 172.21.15.102:0/1720046915 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f39100198c0 con 0x7f3928067c30 2024-09-13T17:42:03.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.300+0000 7f392e280640 1 -- 172.21.15.102:0/1720046915 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f392811fac0 con 0x7f3928067c30 2024-09-13T17:42:03.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.301+0000 7f3924ff9640 1 -- 172.21.15.102:0/1720046915 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1198 (secure 0 0 0) 0x7f3910017020 con 0x7f3928067c30 2024-09-13T17:42:03.303 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:42:03.303 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4210,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6817/3443280887","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":3443280887}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-09-13T17:42:03.303 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 2 2024-09-13T17:42:03.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.303+0000 7f392e280640 1 -- 172.21.15.102:0/1720046915 >> v1:172.21.15.102:6816/34937 conn(0x7f3908031160 legacy=0x7f3908033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:03.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.303+0000 7f392e280640 1 -- 172.21.15.102:0/1720046915 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3928067c30 msgr2=0x7f392811d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:03.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.303+0000 7f392e280640 1 --2- 172.21.15.102:0/1720046915 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3928067c30 0x7f392811d310 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f392806a490 tx=0x7f3910007e10 comp rx=0 tx=0).stop 2024-09-13T17:42:03.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.303+0000 7f392e280640 1 -- 172.21.15.102:0/1720046915 shutdown_connections 2024-09-13T17:42:03.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.303+0000 7f392e280640 1 --2- 172.21.15.102:0/1720046915 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3928098ef0 0x7f392811d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.303+0000 7f392e280640 1 --2- 172.21.15.102:0/1720046915 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3928067c30 0x7f392811d310 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.303+0000 7f392e280640 1 --2- 172.21.15.102:0/1720046915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3928067130 0x7f3928120940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.303+0000 7f392e280640 1 -- 172.21.15.102:0/1720046915 >> 172.21.15.102:0/1720046915 conn(0x7f392805db50 msgr2=0x7f392806ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:03.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.303+0000 7f392e280640 1 -- 172.21.15.102:0/1720046915 shutdown_connections 2024-09-13T17:42:03.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.303+0000 7f392e280640 1 -- 172.21.15.102:0/1720046915 wait complete. 2024-09-13T17:42:03.315 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:03.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.406+0000 7fc739f9d640 1 Processor -- start 2024-09-13T17:42:03.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.406+0000 7fc739f9d640 1 -- start start 2024-09-13T17:42:03.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.407+0000 7fc739f9d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc734067a40 0x7fc73412f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:03.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.407+0000 7fc739f9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc73412fab0 0x7fc73412fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:03.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.407+0000 7fc739f9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc73409e7e0 0x7fc73409ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:03.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.407+0000 7fc739f9d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc734067e10 con 0x7fc734067a40 2024-09-13T17:42:03.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.407+0000 7fc739f9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7340690b0 con 0x7fc73412fab0 2024-09-13T17:42:03.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.407+0000 7fc739f9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc73412f340 con 0x7fc73409e7e0 2024-09-13T17:42:03.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.407+0000 7fc732ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc73412fab0 0x7fc73412fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:03.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.407+0000 7fc733fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc73409e7e0 0x7fc73409ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:03.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.407+0000 7fc7337fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc734067a40 0x7fc73412f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:03.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.407+0000 7fc733fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc73409e7e0 0x7fc73409ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:41866/0 (socket says 172.21.15.102:41866) 2024-09-13T17:42:03.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.407+0000 7fc732ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc73412fab0 0x7fc73412fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:54752/0 (socket says 172.21.15.102:54752) 2024-09-13T17:42:03.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.407+0000 7fc733fff640 1 -- 172.21.15.102:0/4281318243 learned_addr learned my addr 172.21.15.102:0/4281318243 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:03.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.407+0000 7fc7337fe640 1 -- 172.21.15.102:0/4281318243 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc73409e7e0 msgr2=0x7fc73409ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:03.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.407+0000 7fc7337fe640 1 --2- 172.21.15.102:0/4281318243 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc73409e7e0 0x7fc73409ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.407+0000 7fc7337fe640 1 -- 172.21.15.102:0/4281318243 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc73412fab0 msgr2=0x7fc73412fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:03.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.407+0000 7fc7337fe640 1 --2- 172.21.15.102:0/4281318243 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc73412fab0 0x7fc73412fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.407+0000 7fc7337fe640 1 -- 172.21.15.102:0/4281318243 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc734068240 con 0x7fc734067a40 2024-09-13T17:42:03.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.407+0000 7fc733fff640 1 --2- 172.21.15.102:0/4281318243 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc73409e7e0 0x7fc73409ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:03.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.407+0000 7fc732ffd640 1 --2- 172.21.15.102:0/4281318243 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc73412fab0 0x7fc73412fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:03.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.408+0000 7fc7337fe640 1 --2- 172.21.15.102:0/4281318243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc734067a40 0x7fc73412f570 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc724009850 tx=0x7fc7240090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240efeed98c9e170 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:03.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.408+0000 7fc7327fc640 1 -- 172.21.15.102:0/4281318243 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72402abc0 con 0x7fc734067a40 2024-09-13T17:42:03.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.408+0000 7fc7327fc640 1 -- 172.21.15.102:0/4281318243 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72402ad20 con 0x7fc734067a40 2024-09-13T17:42:03.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.408+0000 7fc7327fc640 1 -- 172.21.15.102:0/4281318243 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc724032670 con 0x7fc734067a40 2024-09-13T17:42:03.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.408+0000 7fc739f9d640 1 -- 172.21.15.102:0/4281318243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc734067a40 msgr2=0x7fc73412f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:03.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.408+0000 7fc739f9d640 1 --2- 172.21.15.102:0/4281318243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc734067a40 0x7fc73412f570 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc724009850 tx=0x7fc7240090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.408+0000 7fc739f9d640 1 -- 172.21.15.102:0/4281318243 shutdown_connections 2024-09-13T17:42:03.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.408+0000 7fc739f9d640 1 --2- 172.21.15.102:0/4281318243 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc73409e7e0 0x7fc73409ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.408+0000 7fc739f9d640 1 --2- 172.21.15.102:0/4281318243 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc73412fab0 0x7fc73412fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.408+0000 7fc739f9d640 1 --2- 172.21.15.102:0/4281318243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc734067a40 0x7fc73412f570 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.408+0000 7fc739f9d640 1 -- 172.21.15.102:0/4281318243 >> 172.21.15.102:0/4281318243 conn(0x7fc7340a13a0 msgr2=0x7fc734067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:03.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.409+0000 7fc739f9d640 1 -- 172.21.15.102:0/4281318243 shutdown_connections 2024-09-13T17:42:03.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.409+0000 7fc739f9d640 1 -- 172.21.15.102:0/4281318243 wait complete. 2024-09-13T17:42:03.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.409+0000 7fc739f9d640 1 Processor -- start 2024-09-13T17:42:03.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.409+0000 7fc739f9d640 1 -- start start 2024-09-13T17:42:03.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.410+0000 7fc739f9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc734067a40 0x7fc73411e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:03.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.410+0000 7fc739f9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc73409e7e0 0x7fc73411ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:03.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.410+0000 7fc739f9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc73412fab0 0x7fc73411b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:03.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.410+0000 7fc739f9d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73411c2b0 con 0x7fc73409e7e0 2024-09-13T17:42:03.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.410+0000 7fc739f9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc73409dd90 con 0x7fc734067a40 2024-09-13T17:42:03.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.410+0000 7fc739f9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc734068660 con 0x7fc73412fab0 2024-09-13T17:42:03.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.410+0000 7fc7337fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc734067a40 0x7fc73411e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:03.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.410+0000 7fc732ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc73409e7e0 0x7fc73411ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:03.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.410+0000 7fc733fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc73412fab0 0x7fc73411b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:03.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.410+0000 7fc7337fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc734067a40 0x7fc73411e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:54768/0 (socket says 172.21.15.102:54768) 2024-09-13T17:42:03.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.410+0000 7fc7337fe640 1 -- 172.21.15.102:0/2860026039 learned_addr learned my addr 172.21.15.102:0/2860026039 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:03.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.410+0000 7fc732ffd640 1 -- 172.21.15.102:0/2860026039 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc73412fab0 msgr2=0x7fc73411b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:03.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.410+0000 7fc732ffd640 1 --2- 172.21.15.102:0/2860026039 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc73412fab0 0x7fc73411b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.410+0000 7fc732ffd640 1 -- 172.21.15.102:0/2860026039 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc734067a40 msgr2=0x7fc73411e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:03.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.410+0000 7fc732ffd640 1 --2- 172.21.15.102:0/2860026039 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc734067a40 0x7fc73411e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.410+0000 7fc732ffd640 1 -- 172.21.15.102:0/2860026039 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc72c009ae0 con 0x7fc73409e7e0 2024-09-13T17:42:03.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.410+0000 7fc7337fe640 1 --2- 172.21.15.102:0/2860026039 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc734067a40 0x7fc73411e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:03.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.410+0000 7fc732ffd640 1 --2- 172.21.15.102:0/2860026039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc73409e7e0 0x7fc73411ac90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc72800ea50 tx=0x7fc7280090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:03.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.410+0000 7fc71b7fe640 1 -- 172.21.15.102:0/2860026039 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc728009670 con 0x7fc73409e7e0 2024-09-13T17:42:03.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.411+0000 7fc71b7fe640 1 -- 172.21.15.102:0/2860026039 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72800ca10 con 0x7fc73409e7e0 2024-09-13T17:42:03.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.411+0000 7fc739f9d640 1 -- 172.21.15.102:0/2860026039 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc728009390 con 0x7fc73409e7e0 2024-09-13T17:42:03.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.411+0000 7fc71b7fe640 1 -- 172.21.15.102:0/2860026039 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72800cdb0 con 0x7fc73409e7e0 2024-09-13T17:42:03.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.411+0000 7fc739f9d640 1 -- 172.21.15.102:0/2860026039 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc724009390 con 0x7fc73409e7e0 2024-09-13T17:42:03.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.411+0000 7fc71b7fe640 1 -- 172.21.15.102:0/2860026039 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fc728013680 con 0x7fc73409e7e0 2024-09-13T17:42:03.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.411+0000 7fc71b7fe640 1 -- 172.21.15.102:0/2860026039 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7fc728042050 con 0x7fc73409e7e0 2024-09-13T17:42:03.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.412+0000 7fc739f9d640 1 -- 172.21.15.102:0/2860026039 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc734064000 con 0x7fc73409e7e0 2024-09-13T17:42:03.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.416+0000 7fc71b7fe640 1 -- 172.21.15.102:0/2860026039 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fc728023ad0 con 0x7fc73409e7e0 2024-09-13T17:42:03.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.570+0000 7fc739f9d640 1 -- 172.21.15.102:0/2860026039 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc73405e0d0 con 0x7fc73409e7e0 2024-09-13T17:42:03.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.570+0000 7fc71b7fe640 1 -- 172.21.15.102:0/2860026039 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+8201 (secure 0 0 0) 0x7fc7280232f0 con 0x7fc73409e7e0 2024-09-13T17:42:03.572 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:42:03.572 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":16,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:42:02.123403+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:03.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.572+0000 7fc739f9d640 1 -- 172.21.15.102:0/2860026039 >> v1:172.21.15.102:6816/34937 conn(0x7fc720031160 legacy=0x7fc720033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:03.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.572+0000 7fc739f9d640 1 -- 172.21.15.102:0/2860026039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc73409e7e0 msgr2=0x7fc73411ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:03.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.572+0000 7fc739f9d640 1 --2- 172.21.15.102:0/2860026039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc73409e7e0 0x7fc73411ac90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc72800ea50 tx=0x7fc7280090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.572+0000 7fc739f9d640 1 -- 172.21.15.102:0/2860026039 shutdown_connections 2024-09-13T17:42:03.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.572+0000 7fc739f9d640 1 --2- 172.21.15.102:0/2860026039 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc73412fab0 0x7fc73411b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.572+0000 7fc739f9d640 1 --2- 172.21.15.102:0/2860026039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc73409e7e0 0x7fc73411ac90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.572+0000 7fc739f9d640 1 --2- 172.21.15.102:0/2860026039 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc734067a40 0x7fc73411e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.572+0000 7fc739f9d640 1 -- 172.21.15.102:0/2860026039 >> 172.21.15.102:0/2860026039 conn(0x7fc7340a13a0 msgr2=0x7fc734130480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:03.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.572+0000 7fc739f9d640 1 -- 172.21.15.102:0/2860026039 shutdown_connections 2024-09-13T17:42:03.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.572+0000 7fc739f9d640 1 -- 172.21.15.102:0/2860026039 wait complete. 2024-09-13T17:42:03.585 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:54.442612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T17:41:59.076782+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 16, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2024-09-13T17:42:03.585 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:42:03.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.679+0000 7f4bece34640 1 Processor -- start 2024-09-13T17:42:03.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.679+0000 7f4bece34640 1 -- start start 2024-09-13T17:42:03.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.680+0000 7f4bece34640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4be80698d0 0x7f4be809ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:03.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.680+0000 7f4bece34640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4be809f4d0 0x7f4be812f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:03.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.680+0000 7f4bece34640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4be809f8a0 0x7f4be812f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:03.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.680+0000 7f4bece34640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be8069ca0 con 0x7f4be80698d0 2024-09-13T17:42:03.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.680+0000 7f4bece34640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be80690d0 con 0x7f4be809f8a0 2024-09-13T17:42:03.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.680+0000 7f4bece34640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4be806a2a0 con 0x7f4be809f4d0 2024-09-13T17:42:03.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.680+0000 7f4bdffff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4be809f4d0 0x7f4be812f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:03.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.680+0000 7f4be6d9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4be809f8a0 0x7f4be812f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:03.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.680+0000 7f4be659c640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4be80698d0 0x7f4be809ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:03.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.680+0000 7f4be6d9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4be809f8a0 0x7f4be812f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:54782/0 (socket says 172.21.15.102:54782) 2024-09-13T17:42:03.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.680+0000 7f4be6d9d640 1 -- 172.21.15.102:0/3560215661 learned_addr learned my addr 172.21.15.102:0/3560215661 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:03.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.680+0000 7f4be659c640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4be80698d0 0x7f4be809ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52370/0 (socket says 172.21.15.102:52370) 2024-09-13T17:42:03.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.680+0000 7f4bdffff640 1 -- 172.21.15.102:0/3560215661 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4be809f8a0 msgr2=0x7f4be812f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:03.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.680+0000 7f4bdffff640 1 --2- 172.21.15.102:0/3560215661 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4be809f8a0 0x7f4be812f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.680+0000 7f4bdffff640 1 -- 172.21.15.102:0/3560215661 >> v2:172.21.15.102:3300/0 conn(0x7f4be80698d0 msgr2=0x7f4be809ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:03.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.680+0000 7f4bdffff640 1 --2- 172.21.15.102:0/3560215661 >> v2:172.21.15.102:3300/0 conn(0x7f4be80698d0 0x7f4be809ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.680+0000 7f4bdffff640 1 -- 172.21.15.102:0/3560215661 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be80a0330 con 0x7f4be809f4d0 2024-09-13T17:42:03.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.680+0000 7f4be659c640 1 --2- 172.21.15.102:0/3560215661 >> v2:172.21.15.102:3300/0 conn(0x7f4be80698d0 0x7f4be809ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:03.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.680+0000 7f4be6d9d640 1 --2- 172.21.15.102:0/3560215661 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4be809f8a0 0x7f4be812f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:03.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.681+0000 7f4bdffff640 1 --2- 172.21.15.102:0/3560215661 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4be809f4d0 0x7f4be812f440 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0009880 tx=0x7f4bd00090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=697a5e541ba5f4b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:03.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.681+0000 7f4be5d9b640 1 -- 172.21.15.102:0/3560215661 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd002abc0 con 0x7f4be809f4d0 2024-09-13T17:42:03.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.681+0000 7f4be5d9b640 1 -- 172.21.15.102:0/3560215661 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd002ad20 con 0x7f4be809f4d0 2024-09-13T17:42:03.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.681+0000 7f4be5d9b640 1 -- 172.21.15.102:0/3560215661 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0032670 con 0x7f4be809f4d0 2024-09-13T17:42:03.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.682+0000 7f4bece34640 1 -- 172.21.15.102:0/3560215661 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4be809f4d0 msgr2=0x7f4be812f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:03.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.682+0000 7f4bece34640 1 --2- 172.21.15.102:0/3560215661 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4be809f4d0 0x7f4be812f440 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0009880 tx=0x7f4bd00090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.682+0000 7f4bece34640 1 -- 172.21.15.102:0/3560215661 shutdown_connections 2024-09-13T17:42:03.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.682+0000 7f4bece34640 1 --2- 172.21.15.102:0/3560215661 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4be809f8a0 0x7f4be812f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.682+0000 7f4bece34640 1 --2- 172.21.15.102:0/3560215661 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4be809f4d0 0x7f4be812f440 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.682+0000 7f4bece34640 1 --2- 172.21.15.102:0/3560215661 >> v2:172.21.15.102:3300/0 conn(0x7f4be80698d0 0x7f4be809ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.682+0000 7f4bece34640 1 -- 172.21.15.102:0/3560215661 >> 172.21.15.102:0/3560215661 conn(0x7f4be8067bb0 msgr2=0x7f4be80a08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:03.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.682+0000 7f4bece34640 1 -- 172.21.15.102:0/3560215661 shutdown_connections 2024-09-13T17:42:03.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.682+0000 7f4bece34640 1 -- 172.21.15.102:0/3560215661 wait complete. 2024-09-13T17:42:03.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.683+0000 7f4bece34640 1 Processor -- start 2024-09-13T17:42:03.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.683+0000 7f4bece34640 1 -- start start 2024-09-13T17:42:03.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.683+0000 7f4bece34640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4be80698d0 0x7f4be811c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:03.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.683+0000 7f4bece34640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4be809f4d0 0x7f4be8118da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:03.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.683+0000 7f4bece34640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4be809f8a0 0x7f4be81192e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:03.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.683+0000 7f4bece34640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be8068710 con 0x7f4be809f4d0 2024-09-13T17:42:03.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.683+0000 7f4bece34640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be811ba80 con 0x7f4be809f8a0 2024-09-13T17:42:03.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.683+0000 7f4bece34640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4be811a110 con 0x7f4be80698d0 2024-09-13T17:42:03.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.683+0000 7f4be659c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4be80698d0 0x7f4be811c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:03.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.683+0000 7f4be659c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4be80698d0 0x7f4be811c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:41898/0 (socket says 172.21.15.102:41898) 2024-09-13T17:42:03.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.683+0000 7f4be659c640 1 -- 172.21.15.102:0/600497338 learned_addr learned my addr 172.21.15.102:0/600497338 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:03.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.683+0000 7f4be6d9d640 1 --2- 172.21.15.102:0/600497338 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4be809f8a0 0x7f4be81192e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:03.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.683+0000 7f4bdffff640 1 --2- 172.21.15.102:0/600497338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4be809f4d0 0x7f4be8118da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:03.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.683+0000 7f4be659c640 1 -- 172.21.15.102:0/600497338 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4be809f8a0 msgr2=0x7f4be81192e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:03.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.683+0000 7f4be659c640 1 --2- 172.21.15.102:0/600497338 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4be809f8a0 0x7f4be81192e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.683+0000 7f4be659c640 1 -- 172.21.15.102:0/600497338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4be809f4d0 msgr2=0x7f4be8118da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:03.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.683+0000 7f4be659c640 1 --2- 172.21.15.102:0/600497338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4be809f4d0 0x7f4be8118da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.683+0000 7f4be659c640 1 -- 172.21.15.102:0/600497338 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd8009390 con 0x7f4be80698d0 2024-09-13T17:42:03.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.683+0000 7f4bdffff640 1 --2- 172.21.15.102:0/600497338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4be809f4d0 0x7f4be8118da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:03.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.683+0000 7f4be6d9d640 1 --2- 172.21.15.102:0/600497338 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4be809f8a0 0x7f4be81192e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:03.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.684+0000 7f4be659c640 1 --2- 172.21.15.102:0/600497338 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4be80698d0 0x7f4be811c520 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4bd800ea70 tx=0x7f4bd80090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:03.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.684+0000 7f4bde7fc640 1 -- 172.21.15.102:0/600497338 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd800ca10 con 0x7f4be80698d0 2024-09-13T17:42:03.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.684+0000 7f4bde7fc640 1 -- 172.21.15.102:0/600497338 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd800cb70 con 0x7f4be80698d0 2024-09-13T17:42:03.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.684+0000 7f4bece34640 1 -- 172.21.15.102:0/600497338 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be0009ae0 con 0x7f4be80698d0 2024-09-13T17:42:03.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.684+0000 7f4bde7fc640 1 -- 172.21.15.102:0/600497338 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8013470 con 0x7f4be80698d0 2024-09-13T17:42:03.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.684+0000 7f4bece34640 1 -- 172.21.15.102:0/600497338 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd0009390 con 0x7f4be80698d0 2024-09-13T17:42:03.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.684+0000 7f4bde7fc640 1 -- 172.21.15.102:0/600497338 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f4bd8013820 con 0x7f4be80698d0 2024-09-13T17:42:03.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.685+0000 7f4bde7fc640 1 -- 172.21.15.102:0/600497338 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f4bd8042080 con 0x7f4be80698d0 2024-09-13T17:42:03.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.685+0000 7f4bece34640 1 -- 172.21.15.102:0/600497338 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be8064600 con 0x7f4be80698d0 2024-09-13T17:42:03.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.689+0000 7f4bde7fc640 1 -- 172.21.15.102:0/600497338 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f4bd8023ad0 con 0x7f4be80698d0 2024-09-13T17:42:03.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.842+0000 7f4bece34640 1 -- 172.21.15.102:0/600497338 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4be805e150 con 0x7f4be80698d0 2024-09-13T17:42:03.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.843+0000 7f4bde7fc640 1 -- 172.21.15.102:0/600497338 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4bd80232f0 con 0x7f4be80698d0 2024-09-13T17:42:03.845 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-09-13T17:42:03.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.845+0000 7f4bece34640 1 -- 172.21.15.102:0/600497338 >> v1:172.21.15.102:6816/34937 conn(0x7f4bc8031160 legacy=0x7f4bc8033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:03.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.845+0000 7f4bece34640 1 -- 172.21.15.102:0/600497338 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4be80698d0 msgr2=0x7f4be811c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:03.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.845+0000 7f4bece34640 1 --2- 172.21.15.102:0/600497338 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4be80698d0 0x7f4be811c520 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4bd800ea70 tx=0x7f4bd80090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.845+0000 7f4bece34640 1 -- 172.21.15.102:0/600497338 shutdown_connections 2024-09-13T17:42:03.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.845+0000 7f4bece34640 1 --2- 172.21.15.102:0/600497338 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4be809f8a0 0x7f4be81192e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.845+0000 7f4bece34640 1 --2- 172.21.15.102:0/600497338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4be809f4d0 0x7f4be8118da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.845+0000 7f4bece34640 1 --2- 172.21.15.102:0/600497338 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4be80698d0 0x7f4be811c520 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.845+0000 7f4bece34640 1 -- 172.21.15.102:0/600497338 >> 172.21.15.102:0/600497338 conn(0x7f4be8067bb0 msgr2=0x7f4be812e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:03.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.845+0000 7f4bece34640 1 -- 172.21.15.102:0/600497338 shutdown_connections 2024-09-13T17:42:03.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.845+0000 7f4bece34640 1 -- 172.21.15.102:0/600497338 wait complete. 2024-09-13T17:42:03.857 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-13T17:42:03.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.950+0000 7f796be4f640 1 Processor -- start 2024-09-13T17:42:03.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.950+0000 7f796be4f640 1 -- start start 2024-09-13T17:42:03.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.951+0000 7f796be4f640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7964096a70 0x7f79640987c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:03.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.951+0000 7f796be4f640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f7964098d00 0x7f796412c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:03.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.951+0000 7f796be4f640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7964099110 0x7f796412c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:03.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.951+0000 7f796be4f640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7964069c30 con 0x7f7964098d00 2024-09-13T17:42:03.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.951+0000 7f796be4f640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f796406ed00 con 0x7f7964099110 2024-09-13T17:42:03.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.951+0000 7f796be4f640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79640999e0 con 0x7f7964096a70 2024-09-13T17:42:03.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.951+0000 7f79693ea640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f7964098d00 0x7f796412c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:03.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.951+0000 7f796a3ec640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7964099110 0x7f796412c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:03.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.951+0000 7f7969beb640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7964096a70 0x7f79640987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:03.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.951+0000 7f79693ea640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f7964098d00 0x7f796412c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52384/0 (socket says 172.21.15.102:52384) 2024-09-13T17:42:03.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.951+0000 7f7969beb640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7964096a70 0x7f79640987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:41912/0 (socket says 172.21.15.102:41912) 2024-09-13T17:42:03.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.951+0000 7f796a3ec640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7964099110 0x7f796412c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:54804/0 (socket says 172.21.15.102:54804) 2024-09-13T17:42:03.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.951+0000 7f79693ea640 1 -- 172.21.15.102:0/273469942 learned_addr learned my addr 172.21.15.102:0/273469942 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:03.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.951+0000 7f79693ea640 1 -- 172.21.15.102:0/273469942 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7964096a70 msgr2=0x7f79640987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:03.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.951+0000 7f79693ea640 1 --2- 172.21.15.102:0/273469942 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7964096a70 0x7f79640987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.951+0000 7f79693ea640 1 -- 172.21.15.102:0/273469942 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7964099110 msgr2=0x7f796412c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:03.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.951+0000 7f79693ea640 1 --2- 172.21.15.102:0/273469942 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7964099110 0x7f796412c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.951+0000 7f79693ea640 1 -- 172.21.15.102:0/273469942 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79640997a0 con 0x7f7964098d00 2024-09-13T17:42:03.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.951+0000 7f7969beb640 1 --2- 172.21.15.102:0/273469942 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7964096a70 0x7f79640987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:03.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.951+0000 7f796a3ec640 1 --2- 172.21.15.102:0/273469942 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7964099110 0x7f796412c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:03.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.952+0000 7f79693ea640 1 --2- 172.21.15.102:0/273469942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7964098d00 0x7f796412c2e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f795400ba00 tx=0x7f79540090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3610b5cd2ddd70a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:03.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.952+0000 7f7968be9640 1 -- 172.21.15.102:0/273469942 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795402f510 con 0x7f7964098d00 2024-09-13T17:42:03.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.952+0000 7f7968be9640 1 -- 172.21.15.102:0/273469942 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795402d050 con 0x7f7964098d00 2024-09-13T17:42:03.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.952+0000 7f7968be9640 1 -- 172.21.15.102:0/273469942 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795402f8e0 con 0x7f7964098d00 2024-09-13T17:42:03.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.953+0000 7f796be4f640 1 -- 172.21.15.102:0/273469942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7964098d00 msgr2=0x7f796412c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:03.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.953+0000 7f796be4f640 1 --2- 172.21.15.102:0/273469942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7964098d00 0x7f796412c2e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f795400ba00 tx=0x7f79540090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.953+0000 7f796be4f640 1 -- 172.21.15.102:0/273469942 shutdown_connections 2024-09-13T17:42:03.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.953+0000 7f796be4f640 1 --2- 172.21.15.102:0/273469942 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7964099110 0x7f796412c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.953+0000 7f796be4f640 1 --2- 172.21.15.102:0/273469942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7964098d00 0x7f796412c2e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.953+0000 7f796be4f640 1 --2- 172.21.15.102:0/273469942 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7964096a70 0x7f79640987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.953+0000 7f796be4f640 1 -- 172.21.15.102:0/273469942 >> 172.21.15.102:0/273469942 conn(0x7f796409e720 msgr2=0x7f796409eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:03.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.953+0000 7f796be4f640 1 -- 172.21.15.102:0/273469942 shutdown_connections 2024-09-13T17:42:03.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.953+0000 7f796be4f640 1 -- 172.21.15.102:0/273469942 wait complete. 2024-09-13T17:42:03.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.954+0000 7f796be4f640 1 Processor -- start 2024-09-13T17:42:03.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.954+0000 7f796be4f640 1 -- start start 2024-09-13T17:42:03.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.954+0000 7f796be4f640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7964096a70 0x7f796413a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:03.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.954+0000 7f796be4f640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7964098d00 0x7f796413ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:03.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.954+0000 7f796be4f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7964099110 0x7f7964115280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:03.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.954+0000 7f796be4f640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7964117200 con 0x7f7964099110 2024-09-13T17:42:03.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.954+0000 7f796be4f640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79641188f0 con 0x7f7964098d00 2024-09-13T17:42:03.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.954+0000 7f796be4f640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f796412f610 con 0x7f7964096a70 2024-09-13T17:42:03.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.954+0000 7f7969beb640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7964096a70 0x7f796413a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:03.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.954+0000 7f79693ea640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7964098d00 0x7f796413ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:03.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.954+0000 7f796a3ec640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7964099110 0x7f7964115280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:03.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.954+0000 7f7969beb640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7964096a70 0x7f796413a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:41926/0 (socket says 172.21.15.102:41926) 2024-09-13T17:42:03.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.954+0000 7f7969beb640 1 -- 172.21.15.102:0/3989037717 learned_addr learned my addr 172.21.15.102:0/3989037717 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:03.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.954+0000 7f7969beb640 1 -- 172.21.15.102:0/3989037717 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7964098d00 msgr2=0x7f796413ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:03.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.954+0000 7f7969beb640 1 --2- 172.21.15.102:0/3989037717 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7964098d00 0x7f796413ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.954+0000 7f7969beb640 1 -- 172.21.15.102:0/3989037717 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7964099110 msgr2=0x7f7964115280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:03.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.954+0000 7f7969beb640 1 --2- 172.21.15.102:0/3989037717 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7964099110 0x7f7964115280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.954+0000 7f7969beb640 1 -- 172.21.15.102:0/3989037717 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796000d830 con 0x7f7964096a70 2024-09-13T17:42:03.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.954+0000 7f796a3ec640 1 --2- 172.21.15.102:0/3989037717 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7964099110 0x7f7964115280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:03.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.955+0000 7f7969beb640 1 --2- 172.21.15.102:0/3989037717 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7964096a70 0x7f796413a8c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f796000deb0 tx=0x7f796000f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:03.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.955+0000 7f7959ffb640 1 -- 172.21.15.102:0/3989037717 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796000d2e0 con 0x7f7964096a70 2024-09-13T17:42:03.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.955+0000 7f7959ffb640 1 -- 172.21.15.102:0/3989037717 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7960013040 con 0x7f7964096a70 2024-09-13T17:42:03.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.955+0000 7f7959ffb640 1 -- 172.21.15.102:0/3989037717 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7960010d50 con 0x7f7964096a70 2024-09-13T17:42:03.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.955+0000 7f796be4f640 1 -- 172.21.15.102:0/3989037717 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f795c009a20 con 0x7f7964096a70 2024-09-13T17:42:03.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.955+0000 7f796be4f640 1 -- 172.21.15.102:0/3989037717 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f795400b510 con 0x7f7964096a70 2024-09-13T17:42:03.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.956+0000 7f7959ffb640 1 -- 172.21.15.102:0/3989037717 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f7960018690 con 0x7f7964096a70 2024-09-13T17:42:03.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.956+0000 7f7959ffb640 1 -- 172.21.15.102:0/3989037717 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f7960045cd0 con 0x7f7964096a70 2024-09-13T17:42:03.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.956+0000 7f796be4f640 1 -- 172.21.15.102:0/3989037717 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7964118a30 con 0x7f7964096a70 2024-09-13T17:42:03.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:03.960+0000 7f7959ffb640 1 -- 172.21.15.102:0/3989037717 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f7964118a30 con 0x7f7964096a70 2024-09-13T17:42:04.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.112+0000 7f796be4f640 1 -- 172.21.15.102:0/3989037717 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7964118a30 con 0x7f7964096a70 2024-09-13T17:42:04.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.112+0000 7f7959ffb640 1 -- 172.21.15.102:0/3989037717 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7964118a30 con 0x7f7964096a70 2024-09-13T17:42:04.114 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-09-13T17:42:04.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.114+0000 7f796be4f640 1 -- 172.21.15.102:0/3989037717 >> v1:172.21.15.102:6816/34937 conn(0x7f7948031190 legacy=0x7f7948033640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:04.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.114+0000 7f796be4f640 1 -- 172.21.15.102:0/3989037717 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7964096a70 msgr2=0x7f796413a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:04.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.114+0000 7f796be4f640 1 --2- 172.21.15.102:0/3989037717 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7964096a70 0x7f796413a8c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f796000deb0 tx=0x7f796000f040 comp rx=0 tx=0).stop 2024-09-13T17:42:04.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.114+0000 7f796be4f640 1 -- 172.21.15.102:0/3989037717 shutdown_connections 2024-09-13T17:42:04.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.114+0000 7f796be4f640 1 --2- 172.21.15.102:0/3989037717 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7964099110 0x7f7964115280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:04.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.114+0000 7f796be4f640 1 --2- 172.21.15.102:0/3989037717 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7964098d00 0x7f796413ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:04.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.114+0000 7f796be4f640 1 --2- 172.21.15.102:0/3989037717 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7964096a70 0x7f796413a8c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:04.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.114+0000 7f796be4f640 1 -- 172.21.15.102:0/3989037717 >> 172.21.15.102:0/3989037717 conn(0x7f796409e720 msgr2=0x7f796409c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:04.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.114+0000 7f796be4f640 1 -- 172.21.15.102:0/3989037717 shutdown_connections 2024-09-13T17:42:04.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.114+0000 7f796be4f640 1 -- 172.21.15.102:0/3989037717 wait complete. 2024-09-13T17:42:04.126 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-13T17:42:04.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.218+0000 7fc77999d640 1 Processor -- start 2024-09-13T17:42:04.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.218+0000 7fc77999d640 1 -- start start 2024-09-13T17:42:04.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.218+0000 7fc77999d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc774068920 0x7fc77409d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:04.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.218+0000 7fc77999d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc77409dc50 0x7fc77409e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:04.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.218+0000 7fc77999d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7740695a0 con 0x7fc77412f310 2024-09-13T17:42:04.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.218+0000 7fc77999d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc774068cf0 con 0x7fc774068920 2024-09-13T17:42:04.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.218+0000 7fc77999d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc77409f090 con 0x7fc77409dc50 2024-09-13T17:42:04.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.218+0000 7fc772ffd640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc774068920 0x7fc77409d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:04.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.218+0000 7fc76bfff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc77409dc50 0x7fc77409e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:04.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.218+0000 7fc7737fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fc77412f310 0x7fc77412f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33534/0 (socket says 172.21.15.102:33534) 2024-09-13T17:42:04.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.218+0000 7fc7737fe640 1 -- 172.21.15.102:0/4076349688 learned_addr learned my addr 172.21.15.102:0/4076349688 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:42:04.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.218+0000 7fc76bfff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc77409dc50 0x7fc77409e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:54830/0 (socket says 172.21.15.102:54830) 2024-09-13T17:42:04.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.219+0000 7fc772ffd640 1 -- 172.21.15.102:0/4076349688 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc77409dc50 msgr2=0x7fc77409e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:04.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.219+0000 7fc772ffd640 1 --2- 172.21.15.102:0/4076349688 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc77409dc50 0x7fc77409e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:04.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.219+0000 7fc772ffd640 1 -- 172.21.15.102:0/4076349688 >> v1:172.21.15.102:6789/0 conn(0x7fc77412f310 legacy=0x7fc77412f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:04.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.219+0000 7fc772ffd640 1 -- 172.21.15.102:0/4076349688 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77409cce0 con 0x7fc774068920 2024-09-13T17:42:04.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.219+0000 7fc7727fc640 1 -- 172.21.15.102:0/4076349688 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4106071190 0 0) 0x7fc7740695a0 con 0x7fc77412f310 2024-09-13T17:42:04.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.219+0000 7fc76bfff640 1 --2- 172.21.15.102:0/4076349688 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc77409dc50 0x7fc77409e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:04.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.219+0000 7fc772ffd640 1 --2- 172.21.15.102:0/4076349688 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc774068920 0x7fc77409d710 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc764009850 tx=0x7fc7640090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9151c2b0d8453ac5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:04.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.220+0000 7fc7727fc640 1 -- 172.21.15.102:0/4076349688 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76402ad00 con 0x7fc774068920 2024-09-13T17:42:04.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.220+0000 7fc7727fc640 1 -- 172.21.15.102:0/4076349688 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc76402ae60 con 0x7fc774068920 2024-09-13T17:42:04.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.220+0000 7fc7727fc640 1 -- 172.21.15.102:0/4076349688 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc764032670 con 0x7fc774068920 2024-09-13T17:42:04.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.220+0000 7fc77999d640 1 -- 172.21.15.102:0/4076349688 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc774068920 msgr2=0x7fc77409d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:04.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.220+0000 7fc77999d640 1 --2- 172.21.15.102:0/4076349688 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc774068920 0x7fc77409d710 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc764009850 tx=0x7fc7640090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:04.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.220+0000 7fc77999d640 1 -- 172.21.15.102:0/4076349688 shutdown_connections 2024-09-13T17:42:04.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.220+0000 7fc77999d640 1 --2- 172.21.15.102:0/4076349688 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc77409dc50 0x7fc77409e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:04.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.220+0000 7fc77999d640 1 --2- 172.21.15.102:0/4076349688 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc774068920 0x7fc77409d710 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:04.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.220+0000 7fc77999d640 1 -- 172.21.15.102:0/4076349688 >> 172.21.15.102:0/4076349688 conn(0x7fc77406f320 msgr2=0x7fc7740a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:04.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.220+0000 7fc77999d640 1 -- 172.21.15.102:0/4076349688 shutdown_connections 2024-09-13T17:42:04.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.220+0000 7fc77999d640 1 -- 172.21.15.102:0/4076349688 wait complete. 2024-09-13T17:42:04.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.221+0000 7fc77999d640 1 Processor -- start 2024-09-13T17:42:04.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.221+0000 7fc77999d640 1 -- start start 2024-09-13T17:42:04.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.221+0000 7fc77999d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc774068920 0x7fc77411e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:04.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.221+0000 7fc77999d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc77409dc50 0x7fc77411b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:04.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.221+0000 7fc77999d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc77412f310 0x7fc77411b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:04.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.221+0000 7fc77999d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77409fd50 con 0x7fc77409dc50 2024-09-13T17:42:04.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.221+0000 7fc77999d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc77411dcb0 con 0x7fc77412f310 2024-09-13T17:42:04.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.221+0000 7fc77999d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc77411c340 con 0x7fc774068920 2024-09-13T17:42:04.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.221+0000 7fc772ffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc774068920 0x7fc77411e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:04.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.221+0000 7fc7737fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc77412f310 0x7fc77411b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:04.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.222+0000 7fc772ffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc774068920 0x7fc77411e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:41934/0 (socket says 172.21.15.102:41934) 2024-09-13T17:42:04.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.221+0000 7fc76bfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc77409dc50 0x7fc77411b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:04.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.222+0000 7fc772ffd640 1 -- 172.21.15.102:0/2772783677 learned_addr learned my addr 172.21.15.102:0/2772783677 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:04.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.222+0000 7fc772ffd640 1 -- 172.21.15.102:0/2772783677 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc77412f310 msgr2=0x7fc77411b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:04.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.222+0000 7fc772ffd640 1 --2- 172.21.15.102:0/2772783677 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc77412f310 0x7fc77411b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:04.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.222+0000 7fc772ffd640 1 -- 172.21.15.102:0/2772783677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc77409dc50 msgr2=0x7fc77411b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:04.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.222+0000 7fc772ffd640 1 --2- 172.21.15.102:0/2772783677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc77409dc50 0x7fc77411b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:04.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.222+0000 7fc772ffd640 1 -- 172.21.15.102:0/2772783677 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc75c009390 con 0x7fc774068920 2024-09-13T17:42:04.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.222+0000 7fc772ffd640 1 --2- 172.21.15.102:0/2772783677 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc774068920 0x7fc77411e750 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc76402a630 tx=0x7fc764032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:04.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.222+0000 7fc76affd640 1 -- 172.21.15.102:0/2772783677 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc764031700 con 0x7fc774068920 2024-09-13T17:42:04.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.222+0000 7fc76affd640 1 -- 172.21.15.102:0/2772783677 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc764031860 con 0x7fc774068920 2024-09-13T17:42:04.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.222+0000 7fc77999d640 1 -- 172.21.15.102:0/2772783677 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc764009390 con 0x7fc774068920 2024-09-13T17:42:04.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.222+0000 7fc76affd640 1 -- 172.21.15.102:0/2772783677 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc764031c20 con 0x7fc774068920 2024-09-13T17:42:04.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.222+0000 7fc77999d640 1 -- 172.21.15.102:0/2772783677 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77411ac80 con 0x7fc774068920 2024-09-13T17:42:04.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.223+0000 7fc76affd640 1 -- 172.21.15.102:0/2772783677 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fc764039040 con 0x7fc774068920 2024-09-13T17:42:04.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.223+0000 7fc76affd640 1 -- 172.21.15.102:0/2772783677 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7fc764062020 con 0x7fc774068920 2024-09-13T17:42:04.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.223+0000 7fc77999d640 1 -- 172.21.15.102:0/2772783677 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc77411be00 con 0x7fc774068920 2024-09-13T17:42:04.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.227+0000 7fc76affd640 1 -- 172.21.15.102:0/2772783677 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fc764037080 con 0x7fc774068920 2024-09-13T17:42:04.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.382+0000 7fc77999d640 1 -- 172.21.15.102:0/2772783677 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc774064000 con 0x7fc774068920 2024-09-13T17:42:04.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.382+0000 7fc76affd640 1 -- 172.21.15.102:0/2772783677 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+126 (secure 0 0 0) 0x7fc76403f030 con 0x7fc774068920 2024-09-13T17:42:04.384 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:42:04.384 INFO:teuthology.orchestra.run.smithi102.stdout:[ 2024-09-13T17:42:04.384 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-13T17:42:04.384 INFO:teuthology.orchestra.run.smithi102.stdout: "poolnum": 1, 2024-09-13T17:42:04.384 INFO:teuthology.orchestra.run.smithi102.stdout: "poolname": ".mgr" 2024-09-13T17:42:04.384 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-13T17:42:04.384 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-13T17:42:04.384 INFO:teuthology.orchestra.run.smithi102.stdout: "poolnum": 2, 2024-09-13T17:42:04.385 INFO:teuthology.orchestra.run.smithi102.stdout: "poolname": "rbd" 2024-09-13T17:42:04.385 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-13T17:42:04.385 INFO:teuthology.orchestra.run.smithi102.stdout:] 2024-09-13T17:42:04.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.384+0000 7fc77999d640 1 -- 172.21.15.102:0/2772783677 >> v1:172.21.15.102:6816/34937 conn(0x7fc754031160 legacy=0x7fc754033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:04.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.384+0000 7fc77999d640 1 -- 172.21.15.102:0/2772783677 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc774068920 msgr2=0x7fc77411e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:04.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.384+0000 7fc77999d640 1 --2- 172.21.15.102:0/2772783677 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc774068920 0x7fc77411e750 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc76402a630 tx=0x7fc764032000 comp rx=0 tx=0).stop 2024-09-13T17:42:04.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.384+0000 7fc77999d640 1 -- 172.21.15.102:0/2772783677 shutdown_connections 2024-09-13T17:42:04.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.384+0000 7fc77999d640 1 --2- 172.21.15.102:0/2772783677 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc77412f310 0x7fc77411b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:04.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.384+0000 7fc77999d640 1 --2- 172.21.15.102:0/2772783677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc77409dc50 0x7fc77411b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:04.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.384+0000 7fc77999d640 1 --2- 172.21.15.102:0/2772783677 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc774068920 0x7fc77411e750 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:04.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.384+0000 7fc77999d640 1 -- 172.21.15.102:0/2772783677 >> 172.21.15.102:0/2772783677 conn(0x7fc77406f320 msgr2=0x7fc7740a1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:04.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.384+0000 7fc77999d640 1 -- 172.21.15.102:0/2772783677 shutdown_connections 2024-09-13T17:42:04.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.384+0000 7fc77999d640 1 -- 172.21.15.102:0/2772783677 wait complete. 2024-09-13T17:42:04.396 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-13T17:42:04.396 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-13T17:42:04.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.492+0000 7fb2a8a60640 1 Processor -- start 2024-09-13T17:42:04.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.492+0000 7fb2a8a60640 1 -- start start 2024-09-13T17:42:04.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.492+0000 7fb2a8a60640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb2a412f9d0 0x7fb2a412fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:04.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.492+0000 7fb2a8a60640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2a409eef0 0x7fb2a4137800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:04.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.492+0000 7fb2a8a60640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2a4068240 con 0x7fb2a40a0220 2024-09-13T17:42:04.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.492+0000 7fb2a8a60640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2a412f5c0 con 0x7fb2a412f9d0 2024-09-13T17:42:04.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.492+0000 7fb2a8a60640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a4069040 con 0x7fb2a409eef0 2024-09-13T17:42:04.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.492+0000 7fb2a259c640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb2a412f9d0 0x7fb2a412fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:04.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.492+0000 7fb2a2d9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2a409eef0 0x7fb2a4137800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:04.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.492+0000 7fb2a1d9b640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fb2a40a0220 0x7fb2a409e7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33542/0 (socket says 172.21.15.102:33542) 2024-09-13T17:42:04.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.492+0000 7fb2a1d9b640 1 -- 172.21.15.102:0/2768050444 learned_addr learned my addr 172.21.15.102:0/2768050444 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:42:04.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.493+0000 7fb2a159a640 1 -- 172.21.15.102:0/2768050444 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1016032988 0 0) 0x7fb2a4068240 con 0x7fb2a40a0220 2024-09-13T17:42:04.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.493+0000 7fb2a159a640 1 -- 172.21.15.102:0/2768050444 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb28c0034e0 con 0x7fb2a40a0220 2024-09-13T17:42:04.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.493+0000 7fb2a259c640 1 -- 172.21.15.102:0/2768050444 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2a409eef0 msgr2=0x7fb2a4137800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:04.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.493+0000 7fb2a259c640 1 --2- 172.21.15.102:0/2768050444 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2a409eef0 0x7fb2a4137800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:04.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.493+0000 7fb2a259c640 1 -- 172.21.15.102:0/2768050444 >> v1:172.21.15.102:6789/0 conn(0x7fb2a40a0220 legacy=0x7fb2a409e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:04.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.493+0000 7fb2a259c640 1 -- 172.21.15.102:0/2768050444 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a40a06a0 con 0x7fb2a412f9d0 2024-09-13T17:42:04.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.493+0000 7fb2a2d9d640 1 --2- 172.21.15.102:0/2768050444 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2a409eef0 0x7fb2a4137800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:04.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.493+0000 7fb2a259c640 1 --2- 172.21.15.102:0/2768050444 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2a412f9d0 0x7fb2a412fda0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb294009850 tx=0x7fb2940090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed3b9b761405ca8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:04.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.494+0000 7fb2a159a640 1 -- 172.21.15.102:0/2768050444 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29402ad00 con 0x7fb2a412f9d0 2024-09-13T17:42:04.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.494+0000 7fb2a159a640 1 -- 172.21.15.102:0/2768050444 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb29402ae60 con 0x7fb2a412f9d0 2024-09-13T17:42:04.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.494+0000 7fb2a159a640 1 -- 172.21.15.102:0/2768050444 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294032670 con 0x7fb2a412f9d0 2024-09-13T17:42:04.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.494+0000 7fb2a8a60640 1 -- 172.21.15.102:0/2768050444 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2a412f9d0 msgr2=0x7fb2a412fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:04.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.494+0000 7fb2a8a60640 1 --2- 172.21.15.102:0/2768050444 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2a412f9d0 0x7fb2a412fda0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb294009850 tx=0x7fb2940090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:04.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.494+0000 7fb2a8a60640 1 -- 172.21.15.102:0/2768050444 shutdown_connections 2024-09-13T17:42:04.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.494+0000 7fb2a8a60640 1 --2- 172.21.15.102:0/2768050444 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2a409eef0 0x7fb2a4137800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:04.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.494+0000 7fb2a8a60640 1 --2- 172.21.15.102:0/2768050444 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2a412f9d0 0x7fb2a412fda0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:04.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.494+0000 7fb2a8a60640 1 -- 172.21.15.102:0/2768050444 >> 172.21.15.102:0/2768050444 conn(0x7fb2a4067bb0 msgr2=0x7fb2a40a08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:04.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.494+0000 7fb2a8a60640 1 -- 172.21.15.102:0/2768050444 shutdown_connections 2024-09-13T17:42:04.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.494+0000 7fb2a8a60640 1 -- 172.21.15.102:0/2768050444 wait complete. 2024-09-13T17:42:04.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.495+0000 7fb2a8a60640 1 Processor -- start 2024-09-13T17:42:04.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.495+0000 7fb2a8a60640 1 -- start start 2024-09-13T17:42:04.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.495+0000 7fb2a8a60640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2a409eef0 0x7fb2a413eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:04.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.495+0000 7fb2a8a60640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2a40a0220 0x7fb2a413f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:04.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.495+0000 7fb2a8a60640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2a412f9d0 0x7fb2a413f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:04.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.495+0000 7fb2a8a60640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a406c520 con 0x7fb2a40a0220 2024-09-13T17:42:04.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.495+0000 7fb2a8a60640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a411b920 con 0x7fb2a409eef0 2024-09-13T17:42:04.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.495+0000 7fb2a8a60640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a411be80 con 0x7fb2a412f9d0 2024-09-13T17:42:04.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.496+0000 7fb2a1d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2a40a0220 0x7fb2a413f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:04.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.496+0000 7fb2a2d9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2a412f9d0 0x7fb2a413f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:04.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.496+0000 7fb2a259c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2a409eef0 0x7fb2a413eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:04.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.496+0000 7fb2a1d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2a40a0220 0x7fb2a413f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52430/0 (socket says 172.21.15.102:52430) 2024-09-13T17:42:04.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.496+0000 7fb2a2d9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2a412f9d0 0x7fb2a413f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:41956/0 (socket says 172.21.15.102:41956) 2024-09-13T17:42:04.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.496+0000 7fb2a1d9b640 1 -- 172.21.15.102:0/2746773834 learned_addr learned my addr 172.21.15.102:0/2746773834 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:04.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.496+0000 7fb2a2d9d640 1 -- 172.21.15.102:0/2746773834 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2a409eef0 msgr2=0x7fb2a413eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:04.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.496+0000 7fb2a2d9d640 1 --2- 172.21.15.102:0/2746773834 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2a409eef0 0x7fb2a413eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:04.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.496+0000 7fb2a2d9d640 1 -- 172.21.15.102:0/2746773834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2a40a0220 msgr2=0x7fb2a413f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:04.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.496+0000 7fb2a2d9d640 1 --2- 172.21.15.102:0/2746773834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2a40a0220 0x7fb2a413f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:04.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.496+0000 7fb2a2d9d640 1 -- 172.21.15.102:0/2746773834 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb29c009a20 con 0x7fb2a412f9d0 2024-09-13T17:42:04.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.496+0000 7fb2a1d9b640 1 --2- 172.21.15.102:0/2746773834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2a40a0220 0x7fb2a413f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:04.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.496+0000 7fb2a2d9d640 1 --2- 172.21.15.102:0/2746773834 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2a412f9d0 0x7fb2a413f620 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb29c009f10 tx=0x7fb29c00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:04.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.496+0000 7fb2927fc640 1 -- 172.21.15.102:0/2746773834 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c00cdb0 con 0x7fb2a412f9d0 2024-09-13T17:42:04.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.496+0000 7fb2927fc640 1 -- 172.21.15.102:0/2746773834 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb29c00f040 con 0x7fb2a412f9d0 2024-09-13T17:42:04.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.496+0000 7fb2a8a60640 1 -- 172.21.15.102:0/2746773834 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb294009390 con 0x7fb2a412f9d0 2024-09-13T17:42:04.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.496+0000 7fb2927fc640 1 -- 172.21.15.102:0/2746773834 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c0146d0 con 0x7fb2a412f9d0 2024-09-13T17:42:04.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.496+0000 7fb2a8a60640 1 -- 172.21.15.102:0/2746773834 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a411a250 con 0x7fb2a412f9d0 2024-09-13T17:42:04.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.497+0000 7fb2927fc640 1 -- 172.21.15.102:0/2746773834 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fb29c003020 con 0x7fb2a412f9d0 2024-09-13T17:42:04.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.497+0000 7fb2927fc640 1 -- 172.21.15.102:0/2746773834 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7fb29c042700 con 0x7fb2a412f9d0 2024-09-13T17:42:04.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.497+0000 7fb2a8a60640 1 -- 172.21.15.102:0/2746773834 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2a411ba90 con 0x7fb2a412f9d0 2024-09-13T17:42:04.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.502+0000 7fb2927fc640 1 -- 172.21.15.102:0/2746773834 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fb2a411ba90 con 0x7fb2a412f9d0 2024-09-13T17:42:04.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:04.653+0000 7fb2a8a60640 1 -- 172.21.15.102:0/2746773834 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb2a411ba90 con 0x7fb2a412f9d0 2024-09-13T17:42:05.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.283+0000 7fb2927fc640 1 -- 172.21.15.102:0/2746773834 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb2a411ba90 con 0x7fb2a412f9d0 2024-09-13T17:42:05.285 INFO:teuthology.orchestra.run.smithi102.stderr:pool 'cephfs_metadata' created 2024-09-13T17:42:05.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.288+0000 7fb2a8a60640 1 -- 172.21.15.102:0/2746773834 >> v1:172.21.15.102:6816/34937 conn(0x7fb284031160 legacy=0x7fb284033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:05.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.288+0000 7fb2a8a60640 1 -- 172.21.15.102:0/2746773834 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2a412f9d0 msgr2=0x7fb2a413f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:05.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.288+0000 7fb2a8a60640 1 --2- 172.21.15.102:0/2746773834 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2a412f9d0 0x7fb2a413f620 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb29c009f10 tx=0x7fb29c00b040 comp rx=0 tx=0).stop 2024-09-13T17:42:05.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.288+0000 7fb2a8a60640 1 -- 172.21.15.102:0/2746773834 shutdown_connections 2024-09-13T17:42:05.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.288+0000 7fb2a8a60640 1 --2- 172.21.15.102:0/2746773834 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2a412f9d0 0x7fb2a413f620 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:05.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.288+0000 7fb2a8a60640 1 --2- 172.21.15.102:0/2746773834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2a40a0220 0x7fb2a413f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:05.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.288+0000 7fb2a8a60640 1 --2- 172.21.15.102:0/2746773834 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2a409eef0 0x7fb2a413eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:05.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.288+0000 7fb2a8a60640 1 -- 172.21.15.102:0/2746773834 >> 172.21.15.102:0/2746773834 conn(0x7fb2a4067bb0 msgr2=0x7fb2a406dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:05.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.288+0000 7fb2a8a60640 1 -- 172.21.15.102:0/2746773834 shutdown_connections 2024-09-13T17:42:05.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.289+0000 7fb2a8a60640 1 -- 172.21.15.102:0/2746773834 wait complete. 2024-09-13T17:42:05.302 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-13T17:42:05.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.401+0000 7f1c7879d640 1 Processor -- start 2024-09-13T17:42:05.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.401+0000 7f1c7879d640 1 -- start start 2024-09-13T17:42:05.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.402+0000 7f1c7879d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c70098ed0 0x7f1c70131610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:05.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.402+0000 7f1c7879d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1c70131b50 0x7f1c70133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:05.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.402+0000 7f1c7879d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c700999e0 con 0x7f1c70096a70 2024-09-13T17:42:05.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.402+0000 7f1c7879d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c70069c30 con 0x7f1c70131b50 2024-09-13T17:42:05.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.402+0000 7f1c7879d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7006ed00 con 0x7f1c70098ed0 2024-09-13T17:42:05.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.402+0000 7f1c75d38640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c70098ed0 0x7f1c70131610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:05.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.402+0000 7f1c76d3a640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1c70131b50 0x7f1c70133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:05.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.402+0000 7f1c75d38640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c70098ed0 0x7f1c70131610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:54868/0 (socket says 172.21.15.102:54868) 2024-09-13T17:42:05.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.402+0000 7f1c75d38640 1 -- 172.21.15.102:0/5076105 learned_addr learned my addr 172.21.15.102:0/5076105 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:05.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.402+0000 7f1c75d38640 1 -- 172.21.15.102:0/5076105 >> v2:172.21.15.102:3300/0 conn(0x7f1c70131b50 msgr2=0x7f1c70133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:05.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.402+0000 7f1c75d38640 1 --2- 172.21.15.102:0/5076105 >> v2:172.21.15.102:3300/0 conn(0x7f1c70131b50 0x7f1c70133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:05.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.402+0000 7f1c75d38640 1 -- 172.21.15.102:0/5076105 >> v1:172.21.15.102:6789/0 conn(0x7f1c70096a70 legacy=0x7f1c700987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:05.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.402+0000 7f1c75d38640 1 -- 172.21.15.102:0/5076105 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c700997a0 con 0x7f1c70098ed0 2024-09-13T17:42:05.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.402+0000 7f1c75537640 1 -- 172.21.15.102:0/5076105 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4143094210 0 0) 0x7f1c700999e0 con 0x7f1c70096a70 2024-09-13T17:42:05.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.402+0000 7f1c76d3a640 1 --2- 172.21.15.102:0/5076105 >> v2:172.21.15.102:3300/0 conn(0x7f1c70131b50 0x7f1c70133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:05.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.403+0000 7f1c75d38640 1 --2- 172.21.15.102:0/5076105 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c70098ed0 0x7f1c70131610 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1c60009910 tx=0x7f1c600091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89cb2b63cbf0ce25 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:05.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.403+0000 7f1c75537640 1 -- 172.21.15.102:0/5076105 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6002adb0 con 0x7f1c70098ed0 2024-09-13T17:42:05.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.403+0000 7f1c75537640 1 -- 172.21.15.102:0/5076105 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c60032440 con 0x7f1c70098ed0 2024-09-13T17:42:05.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.403+0000 7f1c75537640 1 -- 172.21.15.102:0/5076105 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c600327e0 con 0x7f1c70098ed0 2024-09-13T17:42:05.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.404+0000 7f1c7879d640 1 -- 172.21.15.102:0/5076105 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c70098ed0 msgr2=0x7f1c70131610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:05.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.404+0000 7f1c7879d640 1 --2- 172.21.15.102:0/5076105 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c70098ed0 0x7f1c70131610 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1c60009910 tx=0x7f1c600091d0 comp rx=0 tx=0).stop 2024-09-13T17:42:05.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.404+0000 7f1c7879d640 1 -- 172.21.15.102:0/5076105 shutdown_connections 2024-09-13T17:42:05.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.404+0000 7f1c7879d640 1 --2- 172.21.15.102:0/5076105 >> v2:172.21.15.102:3300/0 conn(0x7f1c70131b50 0x7f1c70133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:05.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.404+0000 7f1c7879d640 1 --2- 172.21.15.102:0/5076105 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c70098ed0 0x7f1c70131610 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:05.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.404+0000 7f1c7879d640 1 -- 172.21.15.102:0/5076105 >> 172.21.15.102:0/5076105 conn(0x7f1c7009e720 msgr2=0x7f1c7009eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:05.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.404+0000 7f1c7879d640 1 -- 172.21.15.102:0/5076105 shutdown_connections 2024-09-13T17:42:05.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.404+0000 7f1c7879d640 1 -- 172.21.15.102:0/5076105 wait complete. 2024-09-13T17:42:05.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.405+0000 7f1c7879d640 1 Processor -- start 2024-09-13T17:42:05.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.405+0000 7f1c7879d640 1 -- start start 2024-09-13T17:42:05.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.405+0000 7f1c7879d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c70096a70 0x7f1c7012fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:05.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.405+0000 7f1c7879d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1c70098ed0 0x7f1c701302c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:05.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.405+0000 7f1c7879d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c70131b50 0x7f1c70130800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:05.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.405+0000 7f1c7879d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c70116780 con 0x7f1c70131b50 2024-09-13T17:42:05.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.405+0000 7f1c7879d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c70116990 con 0x7f1c70096a70 2024-09-13T17:42:05.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.405+0000 7f1c7879d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1c70115d50 con 0x7f1c70098ed0 2024-09-13T17:42:05.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.405+0000 7f1c76539640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c70096a70 0x7f1c7012fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:05.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.405+0000 7f1c76d3a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c70131b50 0x7f1c70130800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:05.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.405+0000 7f1c76539640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c70096a70 0x7f1c7012fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:54876/0 (socket says 172.21.15.102:54876) 2024-09-13T17:42:05.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.405+0000 7f1c75d38640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1c70098ed0 0x7f1c701302c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:05.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.405+0000 7f1c76539640 1 -- 172.21.15.102:0/2665157520 learned_addr learned my addr 172.21.15.102:0/2665157520 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:05.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.405+0000 7f1c76d3a640 1 -- 172.21.15.102:0/2665157520 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1c70098ed0 msgr2=0x7f1c701302c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:05.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.405+0000 7f1c76d3a640 1 --2- 172.21.15.102:0/2665157520 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1c70098ed0 0x7f1c701302c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:05.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.405+0000 7f1c76d3a640 1 -- 172.21.15.102:0/2665157520 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c70096a70 msgr2=0x7f1c7012fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:05.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.405+0000 7f1c76d3a640 1 --2- 172.21.15.102:0/2665157520 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c70096a70 0x7f1c7012fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:05.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.405+0000 7f1c76d3a640 1 -- 172.21.15.102:0/2665157520 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c68009a20 con 0x7f1c70131b50 2024-09-13T17:42:05.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.405+0000 7f1c76539640 1 --2- 172.21.15.102:0/2665157520 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c70096a70 0x7f1c7012fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:05.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.406+0000 7f1c76d3a640 1 --2- 172.21.15.102:0/2665157520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c70131b50 0x7f1c70130800 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009ee0 tx=0x7f1c6800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:05.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.406+0000 7f1c667fc640 1 -- 172.21.15.102:0/2665157520 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6800ce10 con 0x7f1c70131b50 2024-09-13T17:42:05.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.406+0000 7f1c667fc640 1 -- 172.21.15.102:0/2665157520 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6800f040 con 0x7f1c70131b50 2024-09-13T17:42:05.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.406+0000 7f1c667fc640 1 -- 172.21.15.102:0/2665157520 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c680146d0 con 0x7f1c70131b50 2024-09-13T17:42:05.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.406+0000 7f1c7879d640 1 -- 172.21.15.102:0/2665157520 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c60009450 con 0x7f1c70131b50 2024-09-13T17:42:05.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.406+0000 7f1c7879d640 1 -- 172.21.15.102:0/2665157520 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c7009a410 con 0x7f1c70131b50 2024-09-13T17:42:05.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.407+0000 7f1c667fc640 1 -- 172.21.15.102:0/2665157520 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f1c68003020 con 0x7f1c70131b50 2024-09-13T17:42:05.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.407+0000 7f1c667fc640 1 -- 172.21.15.102:0/2665157520 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3206+0+0 (secure 0 0 0) 0x7f1c68042830 con 0x7f1c70131b50 2024-09-13T17:42:05.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.407+0000 7f1c7879d640 1 -- 172.21.15.102:0/2665157520 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c70064000 con 0x7f1c70131b50 2024-09-13T17:42:05.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.411+0000 7f1c667fc640 1 -- 172.21.15.102:0/2665157520 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f1c68018020 con 0x7f1c70131b50 2024-09-13T17:42:05.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:05.565+0000 7f1c7879d640 1 -- 172.21.15.102:0/2665157520 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1c7006dd60 con 0x7f1c70131b50 2024-09-13T17:42:06.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.282+0000 7f1c667fc640 1 -- 172.21.15.102:0/2665157520 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1c68012b30 con 0x7f1c70131b50 2024-09-13T17:42:06.285 INFO:teuthology.orchestra.run.smithi102.stderr:pool 'cephfs_data' created 2024-09-13T17:42:06.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.298+0000 7f1c7879d640 1 -- 172.21.15.102:0/2665157520 >> v1:172.21.15.102:6816/34937 conn(0x7f1c54031160 legacy=0x7f1c54033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:06.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.298+0000 7f1c7879d640 1 -- 172.21.15.102:0/2665157520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c70131b50 msgr2=0x7f1c70130800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:06.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.298+0000 7f1c7879d640 1 --2- 172.21.15.102:0/2665157520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c70131b50 0x7f1c70130800 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009ee0 tx=0x7f1c6800b040 comp rx=0 tx=0).stop 2024-09-13T17:42:06.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.298+0000 7f1c7879d640 1 -- 172.21.15.102:0/2665157520 shutdown_connections 2024-09-13T17:42:06.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.298+0000 7f1c7879d640 1 --2- 172.21.15.102:0/2665157520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c70131b50 0x7f1c70130800 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:06.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.298+0000 7f1c7879d640 1 --2- 172.21.15.102:0/2665157520 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1c70098ed0 0x7f1c701302c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:06.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.298+0000 7f1c7879d640 1 --2- 172.21.15.102:0/2665157520 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1c70096a70 0x7f1c7012fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:06.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.298+0000 7f1c7879d640 1 -- 172.21.15.102:0/2665157520 >> 172.21.15.102:0/2665157520 conn(0x7f1c7009e720 msgr2=0x7f1c7009de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:06.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.301+0000 7f1c7879d640 1 -- 172.21.15.102:0/2665157520 shutdown_connections 2024-09-13T17:42:06.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.301+0000 7f1c7879d640 1 -- 172.21.15.102:0/2665157520 wait complete. 2024-09-13T17:42:06.320 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-13T17:42:06.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.414+0000 7fcebf79d640 1 Processor -- start 2024-09-13T17:42:06.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.415+0000 7fcebf79d640 1 -- start start 2024-09-13T17:42:06.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.415+0000 7fcebf79d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fceb809b340 0x7fceb809b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:06.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.415+0000 7fcebf79d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fceb809bc50 0x7fceb812c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:06.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.415+0000 7fcebf79d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fceb812c640 0x7fceb812cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:06.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.415+0000 7fcebf79d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fceb809cb80 con 0x7fceb809bc50 2024-09-13T17:42:06.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.415+0000 7fcebf79d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceb8069070 con 0x7fceb809b340 2024-09-13T17:42:06.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.415+0000 7fcebf79d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fceb806a240 con 0x7fceb812c640 2024-09-13T17:42:06.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.415+0000 7fcebd539640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fceb809b340 0x7fceb809b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:06.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.415+0000 7fcebcd38640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fceb809bc50 0x7fceb812c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:06.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.415+0000 7fcebd539640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fceb809b340 0x7fceb809b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:54888/0 (socket says 172.21.15.102:54888) 2024-09-13T17:42:06.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.415+0000 7fcebd539640 1 -- 172.21.15.102:0/3752517772 learned_addr learned my addr 172.21.15.102:0/3752517772 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:06.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.415+0000 7fcebdd3a640 1 --2- 172.21.15.102:0/3752517772 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fceb812c640 0x7fceb812cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:06.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.416+0000 7fcebd539640 1 -- 172.21.15.102:0/3752517772 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fceb812c640 msgr2=0x7fceb812cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:06.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.416+0000 7fcebd539640 1 --2- 172.21.15.102:0/3752517772 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fceb812c640 0x7fceb812cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:06.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.416+0000 7fcebd539640 1 -- 172.21.15.102:0/3752517772 >> v2:172.21.15.102:3300/0 conn(0x7fceb809bc50 msgr2=0x7fceb812c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:06.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.416+0000 7fcebd539640 1 --2- 172.21.15.102:0/3752517772 >> v2:172.21.15.102:3300/0 conn(0x7fceb809bc50 0x7fceb812c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:06.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.416+0000 7fcebd539640 1 -- 172.21.15.102:0/3752517772 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb809d240 con 0x7fceb809b340 2024-09-13T17:42:06.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.416+0000 7fcebcd38640 1 --2- 172.21.15.102:0/3752517772 >> v2:172.21.15.102:3300/0 conn(0x7fceb809bc50 0x7fceb812c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:06.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.416+0000 7fcebd539640 1 --2- 172.21.15.102:0/3752517772 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fceb809b340 0x7fceb809b710 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fceb4009910 tx=0x7fceb40091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33be0ed23686b8b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:06.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.416+0000 7fceaffff640 1 -- 172.21.15.102:0/3752517772 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb402adb0 con 0x7fceb809b340 2024-09-13T17:42:06.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.416+0000 7fceaffff640 1 -- 172.21.15.102:0/3752517772 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb4032440 con 0x7fceb809b340 2024-09-13T17:42:06.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.416+0000 7fceaffff640 1 -- 172.21.15.102:0/3752517772 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb40327e0 con 0x7fceb809b340 2024-09-13T17:42:06.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.417+0000 7fcebf79d640 1 -- 172.21.15.102:0/3752517772 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fceb809b340 msgr2=0x7fceb809b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:06.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.417+0000 7fcebf79d640 1 --2- 172.21.15.102:0/3752517772 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fceb809b340 0x7fceb809b710 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fceb4009910 tx=0x7fceb40091d0 comp rx=0 tx=0).stop 2024-09-13T17:42:06.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.417+0000 7fcebf79d640 1 -- 172.21.15.102:0/3752517772 shutdown_connections 2024-09-13T17:42:06.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.417+0000 7fcebf79d640 1 --2- 172.21.15.102:0/3752517772 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fceb812c640 0x7fceb812cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:06.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.417+0000 7fcebf79d640 1 --2- 172.21.15.102:0/3752517772 >> v2:172.21.15.102:3300/0 conn(0x7fceb809bc50 0x7fceb812c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:06.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.417+0000 7fcebf79d640 1 --2- 172.21.15.102:0/3752517772 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fceb809b340 0x7fceb809b710 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:06.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.417+0000 7fcebf79d640 1 -- 172.21.15.102:0/3752517772 >> 172.21.15.102:0/3752517772 conn(0x7fceb8067c00 msgr2=0x7fceb80a0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:06.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.417+0000 7fcebf79d640 1 -- 172.21.15.102:0/3752517772 shutdown_connections 2024-09-13T17:42:06.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.417+0000 7fcebf79d640 1 -- 172.21.15.102:0/3752517772 wait complete. 2024-09-13T17:42:06.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.418+0000 7fcebf79d640 1 Processor -- start 2024-09-13T17:42:06.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.418+0000 7fcebf79d640 1 -- start start 2024-09-13T17:42:06.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.418+0000 7fcebf79d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fceb809b340 0x7fceb811c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:06.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.418+0000 7fcebf79d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fceb809bc50 0x7fceb811c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:06.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.418+0000 7fcebf79d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fceb812c640 0x7fceb811cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:06.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.418+0000 7fcebf79d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb811fd90 con 0x7fceb812c640 2024-09-13T17:42:06.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.418+0000 7fcebf79d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceb811e000 con 0x7fceb809b340 2024-09-13T17:42:06.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.418+0000 7fcebf79d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fceb809e0e0 con 0x7fceb809bc50 2024-09-13T17:42:06.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.418+0000 7fcebd539640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fceb809b340 0x7fceb811c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:06.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.419+0000 7fcebd539640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fceb809b340 0x7fceb811c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:54892/0 (socket says 172.21.15.102:54892) 2024-09-13T17:42:06.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.419+0000 7fcebcd38640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fceb809bc50 0x7fceb811c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:06.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.419+0000 7fcebd539640 1 -- 172.21.15.102:0/283955206 learned_addr learned my addr 172.21.15.102:0/283955206 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:06.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.419+0000 7fcebdd3a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fceb812c640 0x7fceb811cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:06.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.419+0000 7fcebcd38640 1 -- 172.21.15.102:0/283955206 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fceb809b340 msgr2=0x7fceb811c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:06.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.419+0000 7fcebcd38640 1 --2- 172.21.15.102:0/283955206 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fceb809b340 0x7fceb811c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:06.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.419+0000 7fcebcd38640 1 -- 172.21.15.102:0/283955206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fceb812c640 msgr2=0x7fceb811cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:06.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.419+0000 7fcebcd38640 1 --2- 172.21.15.102:0/283955206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fceb812c640 0x7fceb811cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:06.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.419+0000 7fcebcd38640 1 -- 172.21.15.102:0/283955206 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea8009390 con 0x7fceb809bc50 2024-09-13T17:42:06.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.419+0000 7fcebdd3a640 1 --2- 172.21.15.102:0/283955206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fceb812c640 0x7fceb811cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:06.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.419+0000 7fcebd539640 1 --2- 172.21.15.102:0/283955206 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fceb809b340 0x7fceb811c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:06.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.419+0000 7fcebcd38640 1 --2- 172.21.15.102:0/283955206 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fceb809bc50 0x7fceb811c7f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcea800ea20 tx=0x7fcea80090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:06.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.419+0000 7fcead7fa640 1 -- 172.21.15.102:0/283955206 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea800ca10 con 0x7fceb809bc50 2024-09-13T17:42:06.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.419+0000 7fcead7fa640 1 -- 172.21.15.102:0/283955206 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcea800cb70 con 0x7fceb809bc50 2024-09-13T17:42:06.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.419+0000 7fcebf79d640 1 -- 172.21.15.102:0/283955206 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceb4009450 con 0x7fceb809bc50 2024-09-13T17:42:06.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.419+0000 7fcead7fa640 1 -- 172.21.15.102:0/283955206 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea8013470 con 0x7fceb809bc50 2024-09-13T17:42:06.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.419+0000 7fcebf79d640 1 -- 172.21.15.102:0/283955206 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceb811f440 con 0x7fceb809bc50 2024-09-13T17:42:06.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.420+0000 7fcead7fa640 1 -- 172.21.15.102:0/283955206 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fcea80137e0 con 0x7fceb809bc50 2024-09-13T17:42:06.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.420+0000 7fcead7fa640 1 -- 172.21.15.102:0/283955206 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3588+0+0 (secure 0 0 0) 0x7fcea80422f0 con 0x7fceb809bc50 2024-09-13T17:42:06.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.420+0000 7fcebf79d640 1 -- 172.21.15.102:0/283955206 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fceb8064000 con 0x7fceb809bc50 2024-09-13T17:42:06.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.425+0000 7fcead7fa640 1 -- 172.21.15.102:0/283955206 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fcea801b770 con 0x7fceb809bc50 2024-09-13T17:42:06.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:06.609+0000 7fcebf79d640 1 -- 172.21.15.102:0/283955206 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fceb811f980 con 0x7fceb809bc50 2024-09-13T17:42:07.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.306+0000 7fcead7fa640 1 -- 172.21.15.102:0/283955206 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v3) v1 ==== 172+0+0 (secure 0 0 0) 0x7fcea80091f0 con 0x7fceb809bc50 2024-09-13T17:42:07.309 INFO:teuthology.orchestra.run.smithi102.stderr:new fs with metadata pool 3 and data pool 4 2024-09-13T17:42:07.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.315+0000 7fcebf79d640 1 -- 172.21.15.102:0/283955206 >> v1:172.21.15.102:6816/34937 conn(0x7fce9c031190 legacy=0x7fce9c033640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:07.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.315+0000 7fcebf79d640 1 -- 172.21.15.102:0/283955206 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fceb809bc50 msgr2=0x7fceb811c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:07.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.315+0000 7fcebf79d640 1 --2- 172.21.15.102:0/283955206 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fceb809bc50 0x7fceb811c7f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcea800ea20 tx=0x7fcea80090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:07.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.316+0000 7fcebf79d640 1 -- 172.21.15.102:0/283955206 shutdown_connections 2024-09-13T17:42:07.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.316+0000 7fcebf79d640 1 --2- 172.21.15.102:0/283955206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fceb812c640 0x7fceb811cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:07.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.316+0000 7fcebf79d640 1 --2- 172.21.15.102:0/283955206 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fceb809bc50 0x7fceb811c7f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:07.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.316+0000 7fcebf79d640 1 --2- 172.21.15.102:0/283955206 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fceb809b340 0x7fceb811c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:07.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.316+0000 7fcebf79d640 1 -- 172.21.15.102:0/283955206 >> 172.21.15.102:0/283955206 conn(0x7fceb8067c00 msgr2=0x7fceb809f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:07.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.318+0000 7fcebf79d640 1 -- 172.21.15.102:0/283955206 shutdown_connections 2024-09-13T17:42:07.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.318+0000 7fcebf79d640 1 -- 172.21.15.102:0/283955206 wait complete. 2024-09-13T17:42:07.342 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:07.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.436+0000 7f853429f640 1 Processor -- start 2024-09-13T17:42:07.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.436+0000 7f853429f640 1 -- start start 2024-09-13T17:42:07.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.437+0000 7f853429f640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f852c09f6e0 0x7f852c06e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:07.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.437+0000 7f853429f640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f852c12d990 0x7f852c1355d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:07.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.437+0000 7f853429f640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f852c09fe10 con 0x7f852c06e0b0 2024-09-13T17:42:07.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.437+0000 7f853429f640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f852c09fab0 con 0x7f852c12d990 2024-09-13T17:42:07.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.437+0000 7f853429f640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f852c06c070 con 0x7f852c09f6e0 2024-09-13T17:42:07.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.437+0000 7f853283c640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f852c12d990 0x7f852c1355d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:07.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.437+0000 7f853203b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f852c09f6e0 0x7f852c06e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:07.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.437+0000 7f853283c640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f852c12d990 0x7f852c1355d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52488/0 (socket says 172.21.15.102:52488) 2024-09-13T17:42:07.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.437+0000 7f853283c640 1 -- 172.21.15.102:0/606757258 learned_addr learned my addr 172.21.15.102:0/606757258 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:07.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.437+0000 7f853203b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f852c09f6e0 0x7f852c06e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:41978/0 (socket says 172.21.15.102:41978) 2024-09-13T17:42:07.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.437+0000 7f853183a640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f852c06e0b0 0x7f852c12d280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33562/0 (socket says 172.21.15.102:33562) 2024-09-13T17:42:07.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.437+0000 7f853283c640 1 -- 172.21.15.102:0/606757258 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f852c09f6e0 msgr2=0x7f852c06e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:07.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.437+0000 7f853283c640 1 --2- 172.21.15.102:0/606757258 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f852c09f6e0 0x7f852c06e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:07.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.437+0000 7f853283c640 1 -- 172.21.15.102:0/606757258 >> v1:172.21.15.102:6789/0 conn(0x7f852c06e0b0 legacy=0x7f852c12d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:07.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.437+0000 7f853283c640 1 -- 172.21.15.102:0/606757258 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852c0a0440 con 0x7f852c12d990 2024-09-13T17:42:07.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.437+0000 7f853203b640 1 --2- 172.21.15.102:0/606757258 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f852c09f6e0 0x7f852c06e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:07.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.438+0000 7f853283c640 1 --2- 172.21.15.102:0/606757258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f852c12d990 0x7f852c1355d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8524009f20 tx=0x7f8524009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1747e47284e3cdc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:07.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.438+0000 7f8531039640 1 -- 172.21.15.102:0/606757258 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8524031400 con 0x7f852c12d990 2024-09-13T17:42:07.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.438+0000 7f8531039640 1 -- 172.21.15.102:0/606757258 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f852402d050 con 0x7f852c12d990 2024-09-13T17:42:07.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.438+0000 7f8531039640 1 -- 172.21.15.102:0/606757258 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85240317d0 con 0x7f852c12d990 2024-09-13T17:42:07.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.439+0000 7f853429f640 1 -- 172.21.15.102:0/606757258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f852c12d990 msgr2=0x7f852c1355d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:07.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.439+0000 7f853429f640 1 --2- 172.21.15.102:0/606757258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f852c12d990 0x7f852c1355d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8524009f20 tx=0x7f8524009840 comp rx=0 tx=0).stop 2024-09-13T17:42:07.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.439+0000 7f853429f640 1 -- 172.21.15.102:0/606757258 shutdown_connections 2024-09-13T17:42:07.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.439+0000 7f853429f640 1 --2- 172.21.15.102:0/606757258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f852c12d990 0x7f852c1355d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:07.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.439+0000 7f853429f640 1 --2- 172.21.15.102:0/606757258 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f852c09f6e0 0x7f852c06e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:07.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.439+0000 7f853429f640 1 -- 172.21.15.102:0/606757258 >> 172.21.15.102:0/606757258 conn(0x7f852c067bf0 msgr2=0x7f852c0a07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:07.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.439+0000 7f853429f640 1 -- 172.21.15.102:0/606757258 shutdown_connections 2024-09-13T17:42:07.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.439+0000 7f853429f640 1 -- 172.21.15.102:0/606757258 wait complete. 2024-09-13T17:42:07.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.439+0000 7f853429f640 1 Processor -- start 2024-09-13T17:42:07.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.440+0000 7f853429f640 1 -- start start 2024-09-13T17:42:07.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.440+0000 7f853429f640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f852c06e0b0 0x7f852c118dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:07.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.440+0000 7f853429f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f852c09f6e0 0x7f852c119300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:07.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.440+0000 7f853429f640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f852c12d990 0x7f852c13c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:07.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.440+0000 7f853429f640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852c09cf60 con 0x7f852c09f6e0 2024-09-13T17:42:07.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.440+0000 7f853429f640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852c118b10 con 0x7f852c12d990 2024-09-13T17:42:07.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.440+0000 7f853429f640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f852c11be00 con 0x7f852c06e0b0 2024-09-13T17:42:07.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.440+0000 7f853183a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f852c09f6e0 0x7f852c119300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:07.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.440+0000 7f853203b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f852c06e0b0 0x7f852c118dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:07.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.440+0000 7f853183a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f852c09f6e0 0x7f852c119300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52498/0 (socket says 172.21.15.102:52498) 2024-09-13T17:42:07.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.440+0000 7f853203b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f852c06e0b0 0x7f852c118dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:41994/0 (socket says 172.21.15.102:41994) 2024-09-13T17:42:07.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.440+0000 7f853183a640 1 -- 172.21.15.102:0/523716964 learned_addr learned my addr 172.21.15.102:0/523716964 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:07.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.440+0000 7f853283c640 1 --2- 172.21.15.102:0/523716964 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f852c12d990 0x7f852c13c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:07.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.440+0000 7f853183a640 1 -- 172.21.15.102:0/523716964 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f852c06e0b0 msgr2=0x7f852c118dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:07.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.440+0000 7f853183a640 1 --2- 172.21.15.102:0/523716964 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f852c06e0b0 0x7f852c118dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:07.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.440+0000 7f853183a640 1 -- 172.21.15.102:0/523716964 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f852c12d990 msgr2=0x7f852c13c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:07.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.440+0000 7f853183a640 1 --2- 172.21.15.102:0/523716964 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f852c12d990 0x7f852c13c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:07.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.440+0000 7f853183a640 1 -- 172.21.15.102:0/523716964 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8528009390 con 0x7f852c09f6e0 2024-09-13T17:42:07.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.440+0000 7f853203b640 1 --2- 172.21.15.102:0/523716964 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f852c06e0b0 0x7f852c118dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:07.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.440+0000 7f853183a640 1 --2- 172.21.15.102:0/523716964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f852c09f6e0 0x7f852c119300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f851c00c7c0 tx=0x7f851c007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:07.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.441+0000 7f85227fc640 1 -- 172.21.15.102:0/523716964 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c00ce90 con 0x7f852c09f6e0 2024-09-13T17:42:07.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.441+0000 7f85227fc640 1 -- 172.21.15.102:0/523716964 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851c0070f0 con 0x7f852c09f6e0 2024-09-13T17:42:07.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.441+0000 7f853429f640 1 -- 172.21.15.102:0/523716964 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8524009ae0 con 0x7f852c09f6e0 2024-09-13T17:42:07.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.441+0000 7f85227fc640 1 -- 172.21.15.102:0/523716964 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c0074f0 con 0x7f852c09f6e0 2024-09-13T17:42:07.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.441+0000 7f853429f640 1 -- 172.21.15.102:0/523716964 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f852c11a3c0 con 0x7f852c09f6e0 2024-09-13T17:42:07.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.442+0000 7f85227fc640 1 -- 172.21.15.102:0/523716964 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f851c0049b0 con 0x7f852c09f6e0 2024-09-13T17:42:07.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.442+0000 7f85227fc640 1 -- 172.21.15.102:0/523716964 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7f851c041510 con 0x7f852c09f6e0 2024-09-13T17:42:07.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.442+0000 7f853429f640 1 -- 172.21.15.102:0/523716964 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f852c064000 con 0x7f852c09f6e0 2024-09-13T17:42:07.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.446+0000 7f85227fc640 1 -- 172.21.15.102:0/523716964 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f851c010950 con 0x7f852c09f6e0 2024-09-13T17:42:07.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.599+0000 7f853429f640 1 -- 172.21.15.102:0/523716964 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f852c099db0 con 0x7f852c09f6e0 2024-09-13T17:42:07.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.600+0000 7f85227fc640 1 -- 172.21.15.102:0/523716964 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+11375 (secure 0 0 0) 0x7f851c018070 con 0x7f852c09f6e0 2024-09-13T17:42:07.602 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:42:07.602 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":20,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:42:07.375548+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:04.672265+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:05.583134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:07.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.602+0000 7f853429f640 1 -- 172.21.15.102:0/523716964 >> v1:172.21.15.102:6816/34937 conn(0x7f8510031160 legacy=0x7f8510033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:07.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.602+0000 7f853429f640 1 -- 172.21.15.102:0/523716964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f852c09f6e0 msgr2=0x7f852c119300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:07.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.602+0000 7f853429f640 1 --2- 172.21.15.102:0/523716964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f852c09f6e0 0x7f852c119300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f851c00c7c0 tx=0x7f851c007eb0 comp rx=0 tx=0).stop 2024-09-13T17:42:07.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.602+0000 7f853429f640 1 -- 172.21.15.102:0/523716964 shutdown_connections 2024-09-13T17:42:07.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.602+0000 7f853429f640 1 --2- 172.21.15.102:0/523716964 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f852c12d990 0x7f852c13c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:07.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.602+0000 7f853429f640 1 --2- 172.21.15.102:0/523716964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f852c09f6e0 0x7f852c119300 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:07.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.602+0000 7f853429f640 1 --2- 172.21.15.102:0/523716964 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f852c06e0b0 0x7f852c118dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:07.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.602+0000 7f853429f640 1 -- 172.21.15.102:0/523716964 >> 172.21.15.102:0/523716964 conn(0x7f852c067bf0 msgr2=0x7f852c130390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:07.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.602+0000 7f853429f640 1 -- 172.21.15.102:0/523716964 shutdown_connections 2024-09-13T17:42:07.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.602+0000 7f853429f640 1 -- 172.21.15.102:0/523716964 wait complete. 2024-09-13T17:42:07.614 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:07.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.705+0000 7f4124154640 1 Processor -- start 2024-09-13T17:42:07.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.706+0000 7f4124154640 1 -- start start 2024-09-13T17:42:07.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.706+0000 7f4124154640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f411c0a1390 0x7f411c09e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:07.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.706+0000 7f4124154640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f411c09ed20 0x7f411c09f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:07.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.706+0000 7f4124154640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f411c12c090 0x7f411c12c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:07.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.706+0000 7f4124154640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f411c06a2c0 con 0x7f411c09ed20 2024-09-13T17:42:07.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.706+0000 7f4124154640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411c0a0040 con 0x7f411c12c090 2024-09-13T17:42:07.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.706+0000 7f4124154640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f411c09f630 con 0x7f411c0a1390 2024-09-13T17:42:07.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.706+0000 7f4121ef0640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f411c0a1390 0x7f411c09e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:07.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.706+0000 7f41226f1640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f411c12c090 0x7f411c12c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:07.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.706+0000 7f41216ef640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f411c09ed20 0x7f411c09f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:07.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.706+0000 7f41226f1640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f411c12c090 0x7f411c12c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:54898/0 (socket says 172.21.15.102:54898) 2024-09-13T17:42:07.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.706+0000 7f4121ef0640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f411c0a1390 0x7f411c09e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:42006/0 (socket says 172.21.15.102:42006) 2024-09-13T17:42:07.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.706+0000 7f41216ef640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f411c09ed20 0x7f411c09f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52500/0 (socket says 172.21.15.102:52500) 2024-09-13T17:42:07.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.706+0000 7f41226f1640 1 -- 172.21.15.102:0/4044548003 learned_addr learned my addr 172.21.15.102:0/4044548003 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:07.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.706+0000 7f41226f1640 1 -- 172.21.15.102:0/4044548003 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f411c0a1390 msgr2=0x7f411c09e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:07.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.706+0000 7f41226f1640 1 --2- 172.21.15.102:0/4044548003 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f411c0a1390 0x7f411c09e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:07.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.706+0000 7f41226f1640 1 -- 172.21.15.102:0/4044548003 >> v2:172.21.15.102:3300/0 conn(0x7f411c09ed20 msgr2=0x7f411c09f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:07.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.706+0000 7f41226f1640 1 --2- 172.21.15.102:0/4044548003 >> v2:172.21.15.102:3300/0 conn(0x7f411c09ed20 0x7f411c09f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:07.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.706+0000 7f41226f1640 1 -- 172.21.15.102:0/4044548003 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c068df0 con 0x7f411c12c090 2024-09-13T17:42:07.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.707+0000 7f4121ef0640 1 --2- 172.21.15.102:0/4044548003 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f411c0a1390 0x7f411c09e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:07.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.707+0000 7f41216ef640 1 --2- 172.21.15.102:0/4044548003 >> v2:172.21.15.102:3300/0 conn(0x7f411c09ed20 0x7f411c09f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:07.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.707+0000 7f41226f1640 1 --2- 172.21.15.102:0/4044548003 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f411c12c090 0x7f411c12c4c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f410c009f20 tx=0x7f410c009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53ee291b21e3997 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:07.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.707+0000 7f4120eee640 1 -- 172.21.15.102:0/4044548003 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c031400 con 0x7f411c12c090 2024-09-13T17:42:07.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.707+0000 7f4120eee640 1 -- 172.21.15.102:0/4044548003 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410c02d050 con 0x7f411c12c090 2024-09-13T17:42:07.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.707+0000 7f4120eee640 1 -- 172.21.15.102:0/4044548003 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c0317d0 con 0x7f411c12c090 2024-09-13T17:42:07.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.708+0000 7f4124154640 1 -- 172.21.15.102:0/4044548003 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f411c12c090 msgr2=0x7f411c12c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:07.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.708+0000 7f4124154640 1 --2- 172.21.15.102:0/4044548003 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f411c12c090 0x7f411c12c4c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f410c009f20 tx=0x7f410c009840 comp rx=0 tx=0).stop 2024-09-13T17:42:07.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.708+0000 7f4124154640 1 -- 172.21.15.102:0/4044548003 shutdown_connections 2024-09-13T17:42:07.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.708+0000 7f4124154640 1 --2- 172.21.15.102:0/4044548003 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f411c12c090 0x7f411c12c4c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:07.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.708+0000 7f4124154640 1 --2- 172.21.15.102:0/4044548003 >> v2:172.21.15.102:3300/0 conn(0x7f411c09ed20 0x7f411c09f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:07.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.708+0000 7f4124154640 1 --2- 172.21.15.102:0/4044548003 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f411c0a1390 0x7f411c09e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:07.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.708+0000 7f4124154640 1 -- 172.21.15.102:0/4044548003 >> 172.21.15.102:0/4044548003 conn(0x7f411c067bb0 msgr2=0x7f411c0a08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:07.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.708+0000 7f4124154640 1 -- 172.21.15.102:0/4044548003 shutdown_connections 2024-09-13T17:42:07.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.708+0000 7f4124154640 1 -- 172.21.15.102:0/4044548003 wait complete. 2024-09-13T17:42:07.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.709+0000 7f4124154640 1 Processor -- start 2024-09-13T17:42:07.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.709+0000 7f4124154640 1 -- start start 2024-09-13T17:42:07.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.709+0000 7f4124154640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f411c09ed20 0x7f411c11e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:07.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.709+0000 7f4124154640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f411c0a1390 0x7f411c11e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:07.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.709+0000 7f4124154640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f411c12c090 0x7f411c11eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:07.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.709+0000 7f4124154640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c11fd30 con 0x7f411c12c090 2024-09-13T17:42:07.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.709+0000 7f4124154640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411c1200f0 con 0x7f411c0a1390 2024-09-13T17:42:07.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.709+0000 7f4124154640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f411c1217e0 con 0x7f411c09ed20 2024-09-13T17:42:07.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.709+0000 7f4121ef0640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f411c09ed20 0x7f411c11e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:07.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.709+0000 7f41216ef640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f411c0a1390 0x7f411c11e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:07.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.709+0000 7f4121ef0640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f411c09ed20 0x7f411c11e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:42014/0 (socket says 172.21.15.102:42014) 2024-09-13T17:42:07.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.709+0000 7f4121ef0640 1 -- 172.21.15.102:0/3545186194 learned_addr learned my addr 172.21.15.102:0/3545186194 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:07.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.709+0000 7f41226f1640 1 --2- 172.21.15.102:0/3545186194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f411c12c090 0x7f411c11eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:07.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.710+0000 7f41216ef640 1 -- 172.21.15.102:0/3545186194 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f411c09ed20 msgr2=0x7f411c11e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:07.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.710+0000 7f41216ef640 1 --2- 172.21.15.102:0/3545186194 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f411c09ed20 0x7f411c11e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:07.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.710+0000 7f41216ef640 1 -- 172.21.15.102:0/3545186194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f411c12c090 msgr2=0x7f411c11eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:07.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.710+0000 7f41216ef640 1 --2- 172.21.15.102:0/3545186194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f411c12c090 0x7f411c11eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:07.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.710+0000 7f41216ef640 1 -- 172.21.15.102:0/3545186194 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4110009390 con 0x7f411c0a1390 2024-09-13T17:42:07.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.710+0000 7f4121ef0640 1 --2- 172.21.15.102:0/3545186194 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f411c09ed20 0x7f411c11e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:07.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.710+0000 7f41216ef640 1 --2- 172.21.15.102:0/3545186194 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f411c0a1390 0x7f411c11e5c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f411000e970 tx=0x7f41100090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:07.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.710+0000 7f4119ffb640 1 -- 172.21.15.102:0/3545186194 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41100095d0 con 0x7f411c0a1390 2024-09-13T17:42:07.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.710+0000 7f4119ffb640 1 -- 172.21.15.102:0/3545186194 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411000c940 con 0x7f411c0a1390 2024-09-13T17:42:07.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.710+0000 7f4124154640 1 -- 172.21.15.102:0/3545186194 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4114009390 con 0x7f411c0a1390 2024-09-13T17:42:07.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.710+0000 7f4119ffb640 1 -- 172.21.15.102:0/3545186194 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411000cd00 con 0x7f411c0a1390 2024-09-13T17:42:07.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.710+0000 7f4124154640 1 -- 172.21.15.102:0/3545186194 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f410c009ae0 con 0x7f411c0a1390 2024-09-13T17:42:07.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.711+0000 7f4119ffb640 1 -- 172.21.15.102:0/3545186194 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f4110013680 con 0x7f411c0a1390 2024-09-13T17:42:07.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.711+0000 7f4119ffb640 1 -- 172.21.15.102:0/3545186194 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7f41100422f0 con 0x7f411c0a1390 2024-09-13T17:42:07.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.711+0000 7f4124154640 1 -- 172.21.15.102:0/3545186194 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f411c068680 con 0x7f411c0a1390 2024-09-13T17:42:07.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.716+0000 7f4119ffb640 1 -- 172.21.15.102:0/3545186194 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f4110023ad0 con 0x7f411c0a1390 2024-09-13T17:42:07.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.871+0000 7f4124154640 1 -- 172.21.15.102:0/3545186194 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f411c121440 con 0x7f411c0a1390 2024-09-13T17:42:07.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.871+0000 7f4119ffb640 1 -- 172.21.15.102:0/3545186194 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+11375 (secure 0 0 0) 0x7f411c121440 con 0x7f411c0a1390 2024-09-13T17:42:07.873 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:42:07.874 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":20,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:42:07.375548+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:04.672265+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:05.583134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:07.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.873+0000 7f4124154640 1 -- 172.21.15.102:0/3545186194 >> v1:172.21.15.102:6816/34937 conn(0x7f4108031160 legacy=0x7f4108033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:07.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.873+0000 7f4124154640 1 -- 172.21.15.102:0/3545186194 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f411c0a1390 msgr2=0x7f411c11e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:07.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.873+0000 7f4124154640 1 --2- 172.21.15.102:0/3545186194 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f411c0a1390 0x7f411c11e5c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f411000e970 tx=0x7f41100090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:07.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.873+0000 7f4124154640 1 -- 172.21.15.102:0/3545186194 shutdown_connections 2024-09-13T17:42:07.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.873+0000 7f4124154640 1 --2- 172.21.15.102:0/3545186194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f411c12c090 0x7f411c11eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:07.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.873+0000 7f4124154640 1 --2- 172.21.15.102:0/3545186194 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f411c0a1390 0x7f411c11e5c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:07.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.873+0000 7f4124154640 1 --2- 172.21.15.102:0/3545186194 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f411c09ed20 0x7f411c11e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:07.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.873+0000 7f4124154640 1 -- 172.21.15.102:0/3545186194 >> 172.21.15.102:0/3545186194 conn(0x7f411c067bb0 msgr2=0x7f411c06d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:07.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.873+0000 7f4124154640 1 -- 172.21.15.102:0/3545186194 shutdown_connections 2024-09-13T17:42:07.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.873+0000 7f4124154640 1 -- 172.21.15.102:0/3545186194 wait complete. 2024-09-13T17:42:07.885 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-13T17:42:07.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.979+0000 7fa8ec19d640 1 Processor -- start 2024-09-13T17:42:07.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.979+0000 7fa8ec19d640 1 -- start start 2024-09-13T17:42:07.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.979+0000 7fa8ec19d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa8e409a620 0x7fa8e4096c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:07.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.979+0000 7fa8ec19d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa8e4097540 0x7fa8e412d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:07.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.979+0000 7fa8ec19d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8e4098040 con 0x7fa8e4097170 2024-09-13T17:42:07.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.979+0000 7fa8ec19d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8e409a9f0 con 0x7fa8e4097540 2024-09-13T17:42:07.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.979+0000 7fa8ec19d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e40982a0 con 0x7fa8e409a620 2024-09-13T17:42:07.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.979+0000 7fa8ea73a640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa8e4097540 0x7fa8e412d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:07.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.979+0000 7fa8e9738640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fa8e4097170 0x7fa8e412d180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33568/0 (socket says 172.21.15.102:33568) 2024-09-13T17:42:07.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.979+0000 7fa8e9f39640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa8e409a620 0x7fa8e4096c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:07.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.979+0000 7fa8e9738640 1 -- 172.21.15.102:0/626226718 learned_addr learned my addr 172.21.15.102:0/626226718 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:42:07.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.980+0000 7fa8ea73a640 1 -- 172.21.15.102:0/626226718 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa8e409a620 msgr2=0x7fa8e4096c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:07.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.980+0000 7fa8ea73a640 1 --2- 172.21.15.102:0/626226718 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa8e409a620 0x7fa8e4096c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:07.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.980+0000 7fa8ea73a640 1 -- 172.21.15.102:0/626226718 >> v1:172.21.15.102:6789/0 conn(0x7fa8e4097170 legacy=0x7fa8e412d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:07.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.980+0000 7fa8ea73a640 1 -- 172.21.15.102:0/626226718 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e409ad50 con 0x7fa8e4097540 2024-09-13T17:42:07.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.980+0000 7fa8e8f37640 1 -- 172.21.15.102:0/626226718 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3391197702 0 0) 0x7fa8e4098040 con 0x7fa8e4097170 2024-09-13T17:42:07.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.980+0000 7fa8e9f39640 1 --2- 172.21.15.102:0/626226718 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa8e409a620 0x7fa8e4096c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:07.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.980+0000 7fa8ea73a640 1 --2- 172.21.15.102:0/626226718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8e4097540 0x7fa8e412d890 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa8e409aee0 tx=0x7fa8dc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba24f801399b4110 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:07.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.981+0000 7fa8e8f37640 1 -- 172.21.15.102:0/626226718 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc02aea0 con 0x7fa8e4097540 2024-09-13T17:42:07.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.981+0000 7fa8e8f37640 1 -- 172.21.15.102:0/626226718 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8dc02d050 con 0x7fa8e4097540 2024-09-13T17:42:07.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.981+0000 7fa8e8f37640 1 -- 172.21.15.102:0/626226718 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc007780 con 0x7fa8e4097540 2024-09-13T17:42:07.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.981+0000 7fa8ec19d640 1 -- 172.21.15.102:0/626226718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8e4097540 msgr2=0x7fa8e412d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:07.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.981+0000 7fa8ec19d640 1 --2- 172.21.15.102:0/626226718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8e4097540 0x7fa8e412d890 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa8e409aee0 tx=0x7fa8dc009780 comp rx=0 tx=0).stop 2024-09-13T17:42:07.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.981+0000 7fa8ec19d640 1 -- 172.21.15.102:0/626226718 shutdown_connections 2024-09-13T17:42:07.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.981+0000 7fa8ec19d640 1 --2- 172.21.15.102:0/626226718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8e4097540 0x7fa8e412d890 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:07.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.981+0000 7fa8ec19d640 1 --2- 172.21.15.102:0/626226718 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa8e409a620 0x7fa8e4096c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:07.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.981+0000 7fa8ec19d640 1 -- 172.21.15.102:0/626226718 >> 172.21.15.102:0/626226718 conn(0x7fa8e409e6b0 msgr2=0x7fa8e409eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:07.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.981+0000 7fa8ec19d640 1 -- 172.21.15.102:0/626226718 shutdown_connections 2024-09-13T17:42:07.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.981+0000 7fa8ec19d640 1 -- 172.21.15.102:0/626226718 wait complete. 2024-09-13T17:42:07.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.982+0000 7fa8ec19d640 1 Processor -- start 2024-09-13T17:42:07.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.982+0000 7fa8ec19d640 1 -- start start 2024-09-13T17:42:07.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.982+0000 7fa8ec19d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8e4097170 0x7fa8e413caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:07.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.982+0000 7fa8ec19d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa8e4097540 0x7fa8e413cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:07.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.982+0000 7fa8ec19d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa8e409a620 0x7fa8e4118400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:07.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.982+0000 7fa8ec19d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e4069d20 con 0x7fa8e4097170 2024-09-13T17:42:07.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.982+0000 7fa8ec19d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e41317f0 con 0x7fa8e409a620 2024-09-13T17:42:07.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.982+0000 7fa8ec19d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e411b670 con 0x7fa8e4097540 2024-09-13T17:42:07.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.982+0000 7fa8e9f39640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8e4097170 0x7fa8e413caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:07.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.983+0000 7fa8e9738640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa8e4097540 0x7fa8e413cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:07.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.983+0000 7fa8e9f39640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8e4097170 0x7fa8e413caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52526/0 (socket says 172.21.15.102:52526) 2024-09-13T17:42:07.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.983+0000 7fa8e9f39640 1 -- 172.21.15.102:0/1572582868 learned_addr learned my addr 172.21.15.102:0/1572582868 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:07.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.983+0000 7fa8ea73a640 1 --2- 172.21.15.102:0/1572582868 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa8e409a620 0x7fa8e4118400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:07.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.983+0000 7fa8e9738640 1 -- 172.21.15.102:0/1572582868 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa8e409a620 msgr2=0x7fa8e4118400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:07.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.983+0000 7fa8e9738640 1 --2- 172.21.15.102:0/1572582868 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa8e409a620 0x7fa8e4118400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:07.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.983+0000 7fa8e9738640 1 -- 172.21.15.102:0/1572582868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8e4097170 msgr2=0x7fa8e413caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:07.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.983+0000 7fa8e9738640 1 --2- 172.21.15.102:0/1572582868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8e4097170 0x7fa8e413caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:07.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.983+0000 7fa8e9738640 1 -- 172.21.15.102:0/1572582868 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e0009390 con 0x7fa8e4097540 2024-09-13T17:42:07.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.983+0000 7fa8e9f39640 1 --2- 172.21.15.102:0/1572582868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8e4097170 0x7fa8e413caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:07.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.983+0000 7fa8e9738640 1 --2- 172.21.15.102:0/1572582868 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa8e4097540 0x7fa8e413cfe0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400c410 tx=0x7fa8d4007ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:07.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.983+0000 7fa8d9ffb640 1 -- 172.21.15.102:0/1572582868 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4005aa0 con 0x7fa8e4097540 2024-09-13T17:42:07.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.983+0000 7fa8d9ffb640 1 -- 172.21.15.102:0/1572582868 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d4005c20 con 0x7fa8e4097540 2024-09-13T17:42:07.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.983+0000 7fa8ec19d640 1 -- 172.21.15.102:0/1572582868 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8dc009a20 con 0x7fa8e4097540 2024-09-13T17:42:07.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.983+0000 7fa8d9ffb640 1 -- 172.21.15.102:0/1572582868 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d40070d0 con 0x7fa8e4097540 2024-09-13T17:42:07.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.983+0000 7fa8ec19d640 1 -- 172.21.15.102:0/1572582868 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e411a160 con 0x7fa8e4097540 2024-09-13T17:42:07.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.984+0000 7fa8d9ffb640 1 -- 172.21.15.102:0/1572582868 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fa8d4007500 con 0x7fa8e4097540 2024-09-13T17:42:07.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.984+0000 7fa8d9ffb640 1 -- 172.21.15.102:0/1572582868 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7fa8d4041470 con 0x7fa8e4097540 2024-09-13T17:42:07.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.984+0000 7fa8ec19d640 1 -- 172.21.15.102:0/1572582868 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8e406a650 con 0x7fa8e4097540 2024-09-13T17:42:07.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:07.989+0000 7fa8d9ffb640 1 -- 172.21.15.102:0/1572582868 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fa8d4017070 con 0x7fa8e4097540 2024-09-13T17:42:08.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.170+0000 7fa8ec19d640 1 -- 172.21.15.102:0/1572582868 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa8e4064000 con 0x7fa8e4097540 2024-09-13T17:42:08.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.301+0000 7fa8d9ffb640 1 -- 172.21.15.102:0/1572582868 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v4) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa8d401a360 con 0x7fa8e4097540 2024-09-13T17:42:08.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.303+0000 7fa8ec19d640 1 -- 172.21.15.102:0/1572582868 >> v1:172.21.15.102:6816/34937 conn(0x7fa8c8031160 legacy=0x7fa8c8033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:08.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.303+0000 7fa8ec19d640 1 -- 172.21.15.102:0/1572582868 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa8e4097540 msgr2=0x7fa8e413cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:08.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.303+0000 7fa8ec19d640 1 --2- 172.21.15.102:0/1572582868 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa8e4097540 0x7fa8e413cfe0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400c410 tx=0x7fa8d4007ea0 comp rx=0 tx=0).stop 2024-09-13T17:42:08.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.303+0000 7fa8ec19d640 1 -- 172.21.15.102:0/1572582868 shutdown_connections 2024-09-13T17:42:08.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.303+0000 7fa8ec19d640 1 --2- 172.21.15.102:0/1572582868 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa8e409a620 0x7fa8e4118400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:08.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.303+0000 7fa8ec19d640 1 --2- 172.21.15.102:0/1572582868 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa8e4097540 0x7fa8e413cfe0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:08.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.303+0000 7fa8ec19d640 1 --2- 172.21.15.102:0/1572582868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8e4097170 0x7fa8e413caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:08.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.303+0000 7fa8ec19d640 1 -- 172.21.15.102:0/1572582868 >> 172.21.15.102:0/1572582868 conn(0x7fa8e409e6b0 msgr2=0x7fa8e409d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:08.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.303+0000 7fa8ec19d640 1 -- 172.21.15.102:0/1572582868 shutdown_connections 2024-09-13T17:42:08.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.303+0000 7fa8ec19d640 1 -- 172.21.15.102:0/1572582868 wait complete. 2024-09-13T17:42:08.316 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-13T17:42:08.316 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-13T17:42:08.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.410+0000 7f98f94d4640 1 Processor -- start 2024-09-13T17:42:08.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.410+0000 7f98f94d4640 1 -- start start 2024-09-13T17:42:08.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.410+0000 7f98f94d4640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f98f4132700 0x7f98f4134ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:08.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.410+0000 7f98f94d4640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f98f4135020 0x7f98f4137400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:08.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.410+0000 7f98f94d4640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98f40a2840 con 0x7f98f412f8d0 2024-09-13T17:42:08.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.410+0000 7f98f94d4640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98f4096d80 con 0x7f98f4135020 2024-09-13T17:42:08.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.410+0000 7f98f94d4640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f98f40a0e10 con 0x7f98f4132700 2024-09-13T17:42:08.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.410+0000 7f98f27fc640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f98f4132700 0x7f98f4134ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:08.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.410+0000 7f98f37fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f98f4135020 0x7f98f4137400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:08.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.410+0000 7f98f2ffd640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f98f412f8d0 0x7f98f412fca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33572/0 (socket says 172.21.15.102:33572) 2024-09-13T17:42:08.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.410+0000 7f98f2ffd640 1 -- 172.21.15.102:0/2493121467 learned_addr learned my addr 172.21.15.102:0/2493121467 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:42:08.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.410+0000 7f98f27fc640 1 -- 172.21.15.102:0/2493121467 >> v2:172.21.15.102:3300/0 conn(0x7f98f4135020 msgr2=0x7f98f4137400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:08.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.410+0000 7f98f27fc640 1 --2- 172.21.15.102:0/2493121467 >> v2:172.21.15.102:3300/0 conn(0x7f98f4135020 0x7f98f4137400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:08.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.410+0000 7f98f27fc640 1 -- 172.21.15.102:0/2493121467 >> v1:172.21.15.102:6789/0 conn(0x7f98f412f8d0 legacy=0x7f98f412fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:08.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.410+0000 7f98f27fc640 1 -- 172.21.15.102:0/2493121467 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f412f6b0 con 0x7f98f4132700 2024-09-13T17:42:08.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.411+0000 7f98f27fc640 1 --2- 172.21.15.102:0/2493121467 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f98f4132700 0x7f98f4134ae0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f98e8009850 tx=0x7f98e80090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c247b83adc15336 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:08.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.411+0000 7f98f1ffb640 1 -- 172.21.15.102:0/2493121467 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e802ac60 con 0x7f98f4132700 2024-09-13T17:42:08.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.411+0000 7f98f1ffb640 1 -- 172.21.15.102:0/2493121467 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e802adc0 con 0x7f98f4132700 2024-09-13T17:42:08.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.411+0000 7f98f1ffb640 1 -- 172.21.15.102:0/2493121467 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e8032670 con 0x7f98f4132700 2024-09-13T17:42:08.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.412+0000 7f98f94d4640 1 -- 172.21.15.102:0/2493121467 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f98f4132700 msgr2=0x7f98f4134ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:08.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.412+0000 7f98f94d4640 1 --2- 172.21.15.102:0/2493121467 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f98f4132700 0x7f98f4134ae0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f98e8009850 tx=0x7f98e80090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:08.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.412+0000 7f98f94d4640 1 -- 172.21.15.102:0/2493121467 shutdown_connections 2024-09-13T17:42:08.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.412+0000 7f98f94d4640 1 --2- 172.21.15.102:0/2493121467 >> v2:172.21.15.102:3300/0 conn(0x7f98f4135020 0x7f98f4137400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:08.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.412+0000 7f98f94d4640 1 --2- 172.21.15.102:0/2493121467 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f98f4132700 0x7f98f4134ae0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:08.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.412+0000 7f98f94d4640 1 -- 172.21.15.102:0/2493121467 >> 172.21.15.102:0/2493121467 conn(0x7f98f40a3420 msgr2=0x7f98f406e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:08.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.412+0000 7f98f94d4640 1 -- 172.21.15.102:0/2493121467 shutdown_connections 2024-09-13T17:42:08.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.412+0000 7f98f94d4640 1 -- 172.21.15.102:0/2493121467 wait complete. 2024-09-13T17:42:08.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.413+0000 7f98f94d4640 1 Processor -- start 2024-09-13T17:42:08.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.413+0000 7f98f94d4640 1 -- start start 2024-09-13T17:42:08.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.413+0000 7f98f94d4640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f98f412f8d0 0x7f98f409a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:08.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.413+0000 7f98f94d4640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f98f4132700 0x7f98f409ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:08.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.413+0000 7f98f94d4640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98f4135020 0x7f98f409b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:08.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.413+0000 7f98f94d4640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f411a7b0 con 0x7f98f4135020 2024-09-13T17:42:08.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.413+0000 7f98f94d4640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f412f020 con 0x7f98f412f8d0 2024-09-13T17:42:08.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.413+0000 7f98f94d4640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f98f409ba90 con 0x7f98f4132700 2024-09-13T17:42:08.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.413+0000 7f98f27fc640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f98f4132700 0x7f98f409ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:08.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.413+0000 7f98f2ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f98f412f8d0 0x7f98f409a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:08.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.413+0000 7f98f37fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98f4135020 0x7f98f409b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:08.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.413+0000 7f98f27fc640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f98f4132700 0x7f98f409ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:42030/0 (socket says 172.21.15.102:42030) 2024-09-13T17:42:08.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.413+0000 7f98f27fc640 1 -- 172.21.15.102:0/69214167 learned_addr learned my addr 172.21.15.102:0/69214167 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:08.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.413+0000 7f98f2ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f98f412f8d0 0x7f98f409a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:54934/0 (socket says 172.21.15.102:54934) 2024-09-13T17:42:08.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.414+0000 7f98f27fc640 1 -- 172.21.15.102:0/69214167 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f98f412f8d0 msgr2=0x7f98f409a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:08.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.414+0000 7f98f27fc640 1 --2- 172.21.15.102:0/69214167 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f98f412f8d0 0x7f98f409a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:08.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.414+0000 7f98f27fc640 1 -- 172.21.15.102:0/69214167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98f4135020 msgr2=0x7f98f409b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:08.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.414+0000 7f98f27fc640 1 --2- 172.21.15.102:0/69214167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98f4135020 0x7f98f409b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:08.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.414+0000 7f98f27fc640 1 -- 172.21.15.102:0/69214167 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e8009390 con 0x7f98f4132700 2024-09-13T17:42:08.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.414+0000 7f98f37fe640 1 --2- 172.21.15.102:0/69214167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98f4135020 0x7f98f409b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:08.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.414+0000 7f98f27fc640 1 --2- 172.21.15.102:0/69214167 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f98f4132700 0x7f98f409ab60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f98e8009850 tx=0x7f98e8032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:08.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.414+0000 7f98daffd640 1 -- 172.21.15.102:0/69214167 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e803e660 con 0x7f98f4132700 2024-09-13T17:42:08.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.414+0000 7f98daffd640 1 -- 172.21.15.102:0/69214167 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e803e7c0 con 0x7f98f4132700 2024-09-13T17:42:08.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.414+0000 7f98f94d4640 1 -- 172.21.15.102:0/69214167 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98f411a460 con 0x7f98f4132700 2024-09-13T17:42:08.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.414+0000 7f98daffd640 1 -- 172.21.15.102:0/69214167 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e803eb60 con 0x7f98f4132700 2024-09-13T17:42:08.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.414+0000 7f98f94d4640 1 -- 172.21.15.102:0/69214167 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f409c100 con 0x7f98f4132700 2024-09-13T17:42:08.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.415+0000 7f98daffd640 1 -- 172.21.15.102:0/69214167 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f98e8031420 con 0x7f98f4132700 2024-09-13T17:42:08.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.415+0000 7f98daffd640 1 -- 172.21.15.102:0/69214167 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7f98e8061ef0 con 0x7f98f4132700 2024-09-13T17:42:08.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.415+0000 7f98f94d4640 1 -- 172.21.15.102:0/69214167 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98f4064000 con 0x7f98f4132700 2024-09-13T17:42:08.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.419+0000 7f98daffd640 1 -- 172.21.15.102:0/69214167 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f98e8037080 con 0x7f98f4132700 2024-09-13T17:42:08.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:08.605+0000 7f98f94d4640 1 -- 172.21.15.102:0/69214167 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f98f40a21a0 con 0x7f98f4132700 2024-09-13T17:42:09.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.315+0000 7f98daffd640 1 -- 172.21.15.102:0/69214167 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v5) v1 ==== 122+0+0 (secure 0 0 0) 0x7f98e8043020 con 0x7f98f4132700 2024-09-13T17:42:09.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.317+0000 7f98f94d4640 1 -- 172.21.15.102:0/69214167 >> v1:172.21.15.102:6816/34937 conn(0x7f98e0031160 legacy=0x7f98e0033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:09.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.317+0000 7f98f94d4640 1 -- 172.21.15.102:0/69214167 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f98f4132700 msgr2=0x7f98f409ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:09.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.317+0000 7f98f94d4640 1 --2- 172.21.15.102:0/69214167 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f98f4132700 0x7f98f409ab60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f98e8009850 tx=0x7f98e8032000 comp rx=0 tx=0).stop 2024-09-13T17:42:09.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.317+0000 7f98f94d4640 1 -- 172.21.15.102:0/69214167 shutdown_connections 2024-09-13T17:42:09.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.317+0000 7f98f94d4640 1 --2- 172.21.15.102:0/69214167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98f4135020 0x7f98f409b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:09.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.317+0000 7f98f94d4640 1 --2- 172.21.15.102:0/69214167 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f98f4132700 0x7f98f409ab60 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:09.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.317+0000 7f98f94d4640 1 --2- 172.21.15.102:0/69214167 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f98f412f8d0 0x7f98f409a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:09.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.317+0000 7f98f94d4640 1 -- 172.21.15.102:0/69214167 >> 172.21.15.102:0/69214167 conn(0x7f98f40a3420 msgr2=0x7f98f41342e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:09.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.317+0000 7f98f94d4640 1 -- 172.21.15.102:0/69214167 shutdown_connections 2024-09-13T17:42:09.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.317+0000 7f98f94d4640 1 -- 172.21.15.102:0/69214167 wait complete. 2024-09-13T17:42:09.330 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-13T17:42:09.331 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:42:09.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.424+0000 7f7a5439d640 1 Processor -- start 2024-09-13T17:42:09.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.424+0000 7f7a5439d640 1 -- start start 2024-09-13T17:42:09.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.425+0000 7f7a5439d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7a4c09f7f0 0x7f7a4c132650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:09.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.425+0000 7f7a5439d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f7a4c132b90 0x7f7a4c134f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:09.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.425+0000 7f7a5439d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a4c06c910 con 0x7f7a4c06d670 2024-09-13T17:42:09.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.425+0000 7f7a5439d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a4c09fbc0 con 0x7f7a4c132b90 2024-09-13T17:42:09.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.425+0000 7f7a5439d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c06c130 con 0x7f7a4c09f7f0 2024-09-13T17:42:09.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.425+0000 7f7a51938640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7a4c09f7f0 0x7f7a4c132650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:09.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.425+0000 7f7a52139640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f7a4c06d670 0x7f7a4c06e620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33574/0 (socket says 172.21.15.102:33574) 2024-09-13T17:42:09.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.425+0000 7f7a5293a640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f7a4c132b90 0x7f7a4c134f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:09.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.425+0000 7f7a52139640 1 -- 172.21.15.102:0/1491208087 learned_addr learned my addr 172.21.15.102:0/1491208087 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:42:09.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.425+0000 7f7a5293a640 1 -- 172.21.15.102:0/1491208087 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7a4c09f7f0 msgr2=0x7f7a4c132650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:09.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.425+0000 7f7a5293a640 1 --2- 172.21.15.102:0/1491208087 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7a4c09f7f0 0x7f7a4c132650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:09.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.425+0000 7f7a5293a640 1 -- 172.21.15.102:0/1491208087 >> v1:172.21.15.102:6789/0 conn(0x7f7a4c06d670 legacy=0x7f7a4c06e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:09.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.425+0000 7f7a5293a640 1 -- 172.21.15.102:0/1491208087 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4c0a0310 con 0x7f7a4c132b90 2024-09-13T17:42:09.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.425+0000 7f7a51137640 1 -- 172.21.15.102:0/1491208087 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2277417274 0 0) 0x7f7a4c06c910 con 0x7f7a4c06d670 2024-09-13T17:42:09.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.425+0000 7f7a51938640 1 --2- 172.21.15.102:0/1491208087 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7a4c09f7f0 0x7f7a4c132650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:09.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.426+0000 7f7a5293a640 1 --2- 172.21.15.102:0/1491208087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a4c132b90 0x7f7a4c134f70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7a44009850 tx=0x7f7a440090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c25461915a76986 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:09.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.426+0000 7f7a51137640 1 -- 172.21.15.102:0/1491208087 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4402ad00 con 0x7f7a4c132b90 2024-09-13T17:42:09.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.426+0000 7f7a51137640 1 -- 172.21.15.102:0/1491208087 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4402ae60 con 0x7f7a4c132b90 2024-09-13T17:42:09.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.426+0000 7f7a51137640 1 -- 172.21.15.102:0/1491208087 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a44032670 con 0x7f7a4c132b90 2024-09-13T17:42:09.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.427+0000 7f7a5439d640 1 -- 172.21.15.102:0/1491208087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a4c132b90 msgr2=0x7f7a4c134f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:09.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.427+0000 7f7a5439d640 1 --2- 172.21.15.102:0/1491208087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a4c132b90 0x7f7a4c134f70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7a44009850 tx=0x7f7a440090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:09.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.427+0000 7f7a5439d640 1 -- 172.21.15.102:0/1491208087 shutdown_connections 2024-09-13T17:42:09.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.427+0000 7f7a5439d640 1 --2- 172.21.15.102:0/1491208087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a4c132b90 0x7f7a4c134f70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:09.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.427+0000 7f7a5439d640 1 --2- 172.21.15.102:0/1491208087 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7a4c09f7f0 0x7f7a4c132650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:09.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.427+0000 7f7a5439d640 1 -- 172.21.15.102:0/1491208087 >> 172.21.15.102:0/1491208087 conn(0x7f7a4c067bb0 msgr2=0x7f7a4c0a08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:09.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.427+0000 7f7a5439d640 1 -- 172.21.15.102:0/1491208087 shutdown_connections 2024-09-13T17:42:09.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.427+0000 7f7a5439d640 1 -- 172.21.15.102:0/1491208087 wait complete. 2024-09-13T17:42:09.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.428+0000 7f7a5439d640 1 Processor -- start 2024-09-13T17:42:09.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.428+0000 7f7a5439d640 1 -- start start 2024-09-13T17:42:09.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.428+0000 7f7a5439d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7a4c06d670 0x7f7a4c117e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:09.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.428+0000 7f7a5439d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7a4c09f7f0 0x7f7a4c1183c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:09.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.428+0000 7f7a5439d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a4c132b90 0x7f7a4c118900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:09.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.428+0000 7f7a5439d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c119ea0 con 0x7f7a4c132b90 2024-09-13T17:42:09.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.428+0000 7f7a5439d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c119960 con 0x7f7a4c06d670 2024-09-13T17:42:09.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.428+0000 7f7a5439d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c11b2d0 con 0x7f7a4c09f7f0 2024-09-13T17:42:09.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.428+0000 7f7a5293a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a4c132b90 0x7f7a4c118900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:09.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.428+0000 7f7a51938640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7a4c09f7f0 0x7f7a4c1183c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:09.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.428+0000 7f7a52139640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7a4c06d670 0x7f7a4c117e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:09.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.428+0000 7f7a5293a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a4c132b90 0x7f7a4c118900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52558/0 (socket says 172.21.15.102:52558) 2024-09-13T17:42:09.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.428+0000 7f7a5293a640 1 -- 172.21.15.102:0/3695974557 learned_addr learned my addr 172.21.15.102:0/3695974557 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:09.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.429+0000 7f7a52139640 1 -- 172.21.15.102:0/3695974557 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7a4c09f7f0 msgr2=0x7f7a4c1183c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:09.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.429+0000 7f7a52139640 1 --2- 172.21.15.102:0/3695974557 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7a4c09f7f0 0x7f7a4c1183c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:09.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.429+0000 7f7a52139640 1 -- 172.21.15.102:0/3695974557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a4c132b90 msgr2=0x7f7a4c118900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:09.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.429+0000 7f7a52139640 1 --2- 172.21.15.102:0/3695974557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a4c132b90 0x7f7a4c118900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:09.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.429+0000 7f7a52139640 1 -- 172.21.15.102:0/3695974557 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a40009390 con 0x7f7a4c06d670 2024-09-13T17:42:09.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.429+0000 7f7a5293a640 1 --2- 172.21.15.102:0/3695974557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a4c132b90 0x7f7a4c118900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:09.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.429+0000 7f7a51938640 1 --2- 172.21.15.102:0/3695974557 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7a4c09f7f0 0x7f7a4c1183c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:09.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.429+0000 7f7a52139640 1 --2- 172.21.15.102:0/3695974557 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7a4c06d670 0x7f7a4c117e80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c00cb60 tx=0x7f7a3c001950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:09.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.429+0000 7f7a4a7fc640 1 -- 172.21.15.102:0/3695974557 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c00a640 con 0x7f7a4c06d670 2024-09-13T17:42:09.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.429+0000 7f7a4a7fc640 1 -- 172.21.15.102:0/3695974557 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3c00a7c0 con 0x7f7a4c06d670 2024-09-13T17:42:09.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.429+0000 7f7a5439d640 1 -- 172.21.15.102:0/3695974557 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a44009390 con 0x7f7a4c06d670 2024-09-13T17:42:09.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.429+0000 7f7a4a7fc640 1 -- 172.21.15.102:0/3695974557 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c00ac00 con 0x7f7a4c06d670 2024-09-13T17:42:09.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.429+0000 7f7a5439d640 1 -- 172.21.15.102:0/3695974557 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a4c12e020 con 0x7f7a4c06d670 2024-09-13T17:42:09.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.430+0000 7f7a4a7fc640 1 -- 172.21.15.102:0/3695974557 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f7a3c007300 con 0x7f7a4c06d670 2024-09-13T17:42:09.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.430+0000 7f7a4a7fc640 1 -- 172.21.15.102:0/3695974557 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7f7a3c042080 con 0x7f7a4c06d670 2024-09-13T17:42:09.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.430+0000 7f7a5439d640 1 -- 172.21.15.102:0/3695974557 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a4c11b6a0 con 0x7f7a4c06d670 2024-09-13T17:42:09.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.434+0000 7f7a4a7fc640 1 -- 172.21.15.102:0/3695974557 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f7a3c0075b0 con 0x7f7a4c06d670 2024-09-13T17:42:09.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.614+0000 7f7a5439d640 1 -- 172.21.15.102:0/3695974557 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a4c064600 con 0x7f7a4c06d670 2024-09-13T17:42:09.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.614+0000 7f7a4a7fc640 1 -- 172.21.15.102:0/3695974557 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2266 (secure 0 0 0) 0x7f7a3c010770 con 0x7f7a4c06d670 2024-09-13T17:42:09.616 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:42:09.616 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:42:06.611970+0000","modified":"2024-09-13T17:42:09.305093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4210":{"gid":4210,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6817/3443280887","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":3443280887}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-09-13T17:42:09.617 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 5 2024-09-13T17:42:09.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.616+0000 7f7a5439d640 1 -- 172.21.15.102:0/3695974557 >> v1:172.21.15.102:6816/34937 conn(0x7f7a38031160 legacy=0x7f7a38033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:09.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.616+0000 7f7a5439d640 1 -- 172.21.15.102:0/3695974557 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7a4c06d670 msgr2=0x7f7a4c117e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:09.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.616+0000 7f7a5439d640 1 --2- 172.21.15.102:0/3695974557 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7a4c06d670 0x7f7a4c117e80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c00cb60 tx=0x7f7a3c001950 comp rx=0 tx=0).stop 2024-09-13T17:42:09.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.616+0000 7f7a5439d640 1 -- 172.21.15.102:0/3695974557 shutdown_connections 2024-09-13T17:42:09.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.616+0000 7f7a5439d640 1 --2- 172.21.15.102:0/3695974557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a4c132b90 0x7f7a4c118900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:09.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.616+0000 7f7a5439d640 1 --2- 172.21.15.102:0/3695974557 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7a4c09f7f0 0x7f7a4c1183c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:09.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.616+0000 7f7a5439d640 1 --2- 172.21.15.102:0/3695974557 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7a4c06d670 0x7f7a4c117e80 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:09.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.616+0000 7f7a5439d640 1 -- 172.21.15.102:0/3695974557 >> 172.21.15.102:0/3695974557 conn(0x7f7a4c067bb0 msgr2=0x7f7a4c09a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:09.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.616+0000 7f7a5439d640 1 -- 172.21.15.102:0/3695974557 shutdown_connections 2024-09-13T17:42:09.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.616+0000 7f7a5439d640 1 -- 172.21.15.102:0/3695974557 wait complete. 2024-09-13T17:42:09.628 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:09.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.720+0000 7ff01979d640 1 Processor -- start 2024-09-13T17:42:09.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.720+0000 7ff01979d640 1 -- start start 2024-09-13T17:42:09.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.720+0000 7ff01979d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff014068920 0x7ff01409d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:09.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.720+0000 7ff01979d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff01409dc50 0x7ff01409e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:09.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.720+0000 7ff01979d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0140695a0 con 0x7ff01412f310 2024-09-13T17:42:09.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.720+0000 7ff01979d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff014068cf0 con 0x7ff01409dc50 2024-09-13T17:42:09.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.720+0000 7ff01979d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff01409f090 con 0x7ff014068920 2024-09-13T17:42:09.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.720+0000 7ff012ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff014068920 0x7ff01409d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:09.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.720+0000 7ff00a7fc640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff01409dc50 0x7ff01409e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:09.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.721+0000 7ff00a7fc640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff01409dc50 0x7ff01409e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52562/0 (socket says 172.21.15.102:52562) 2024-09-13T17:42:09.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.721+0000 7ff012ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff014068920 0x7ff01409d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:54948/0 (socket says 172.21.15.102:54948) 2024-09-13T17:42:09.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.721+0000 7ff00a7fc640 1 -- 172.21.15.102:0/2556510636 learned_addr learned my addr 172.21.15.102:0/2556510636 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:09.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.721+0000 7ff0137fe640 1 --1- 172.21.15.102:0/2556510636 >> v1:172.21.15.102:6789/0 conn(0x7ff01412f310 0x7ff01412f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33586/0 (socket says 172.21.15.102:33586) 2024-09-13T17:42:09.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.721+0000 7ff012ffd640 1 -- 172.21.15.102:0/2556510636 >> v2:172.21.15.102:3300/0 conn(0x7ff01409dc50 msgr2=0x7ff01409e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:09.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.721+0000 7ff012ffd640 1 --2- 172.21.15.102:0/2556510636 >> v2:172.21.15.102:3300/0 conn(0x7ff01409dc50 0x7ff01409e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:09.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.721+0000 7ff012ffd640 1 -- 172.21.15.102:0/2556510636 >> v1:172.21.15.102:6789/0 conn(0x7ff01412f310 legacy=0x7ff01412f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:09.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.721+0000 7ff012ffd640 1 -- 172.21.15.102:0/2556510636 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff01409cce0 con 0x7ff014068920 2024-09-13T17:42:09.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.721+0000 7ff00a7fc640 1 --2- 172.21.15.102:0/2556510636 >> v2:172.21.15.102:3300/0 conn(0x7ff01409dc50 0x7ff01409e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:09.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.721+0000 7ff012ffd640 1 --2- 172.21.15.102:0/2556510636 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff014068920 0x7ff01409d710 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff004009850 tx=0x7ff0040090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c040f3944f6e7a79 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:09.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.722+0000 7ff0127fc640 1 -- 172.21.15.102:0/2556510636 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00402ad00 con 0x7ff014068920 2024-09-13T17:42:09.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.722+0000 7ff0127fc640 1 -- 172.21.15.102:0/2556510636 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff00402ae60 con 0x7ff014068920 2024-09-13T17:42:09.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.722+0000 7ff0127fc640 1 -- 172.21.15.102:0/2556510636 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff004032670 con 0x7ff014068920 2024-09-13T17:42:09.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.722+0000 7ff01979d640 1 -- 172.21.15.102:0/2556510636 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff014068920 msgr2=0x7ff01409d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:09.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.722+0000 7ff01979d640 1 --2- 172.21.15.102:0/2556510636 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff014068920 0x7ff01409d710 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff004009850 tx=0x7ff0040090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:09.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.722+0000 7ff01979d640 1 -- 172.21.15.102:0/2556510636 shutdown_connections 2024-09-13T17:42:09.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.722+0000 7ff01979d640 1 --2- 172.21.15.102:0/2556510636 >> v2:172.21.15.102:3300/0 conn(0x7ff01409dc50 0x7ff01409e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:09.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.722+0000 7ff01979d640 1 --2- 172.21.15.102:0/2556510636 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff014068920 0x7ff01409d710 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:09.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.722+0000 7ff01979d640 1 -- 172.21.15.102:0/2556510636 >> 172.21.15.102:0/2556510636 conn(0x7ff01406f320 msgr2=0x7ff0140a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:09.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.722+0000 7ff01979d640 1 -- 172.21.15.102:0/2556510636 shutdown_connections 2024-09-13T17:42:09.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.722+0000 7ff01979d640 1 -- 172.21.15.102:0/2556510636 wait complete. 2024-09-13T17:42:09.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.723+0000 7ff01979d640 1 Processor -- start 2024-09-13T17:42:09.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.723+0000 7ff01979d640 1 -- start start 2024-09-13T17:42:09.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.724+0000 7ff01979d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff014068920 0x7ff01411b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:09.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.724+0000 7ff01979d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff01409dc50 0x7ff01411b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:09.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.724+0000 7ff01979d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff01412f310 0x7ff01413ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:09.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.724+0000 7ff01979d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01411add0 con 0x7ff01409dc50 2024-09-13T17:42:09.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.724+0000 7ff01979d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff01411e020 con 0x7ff014068920 2024-09-13T17:42:09.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.724+0000 7ff01979d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff01411cd00 con 0x7ff01412f310 2024-09-13T17:42:09.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.724+0000 7ff012ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff014068920 0x7ff01411b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:09.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.724+0000 7ff00a7fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff01409dc50 0x7ff01411b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:09.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.724+0000 7ff012ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff014068920 0x7ff01411b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:54950/0 (socket says 172.21.15.102:54950) 2024-09-13T17:42:09.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.724+0000 7ff00a7fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff01409dc50 0x7ff01411b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52576/0 (socket says 172.21.15.102:52576) 2024-09-13T17:42:09.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.724+0000 7ff012ffd640 1 -- 172.21.15.102:0/593086414 learned_addr learned my addr 172.21.15.102:0/593086414 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:09.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.724+0000 7ff0137fe640 1 --2- 172.21.15.102:0/593086414 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff01412f310 0x7ff01413ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:09.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.724+0000 7ff012ffd640 1 -- 172.21.15.102:0/593086414 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff01412f310 msgr2=0x7ff01413ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:09.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.724+0000 7ff012ffd640 1 --2- 172.21.15.102:0/593086414 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff01412f310 0x7ff01413ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:09.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.724+0000 7ff012ffd640 1 -- 172.21.15.102:0/593086414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff01409dc50 msgr2=0x7ff01411b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:09.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.724+0000 7ff012ffd640 1 --2- 172.21.15.102:0/593086414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff01409dc50 0x7ff01411b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:09.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.724+0000 7ff012ffd640 1 -- 172.21.15.102:0/593086414 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feffc009450 con 0x7ff014068920 2024-09-13T17:42:09.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.724+0000 7ff00a7fc640 1 --2- 172.21.15.102:0/593086414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff01409dc50 0x7ff01411b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:09.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.724+0000 7ff012ffd640 1 --2- 172.21.15.102:0/593086414 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff014068920 0x7ff01411b020 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff00402a630 tx=0x7ff004032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:09.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.724+0000 7ff00b7fe640 1 -- 172.21.15.102:0/593086414 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff004031700 con 0x7ff014068920 2024-09-13T17:42:09.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.724+0000 7ff00b7fe640 1 -- 172.21.15.102:0/593086414 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff004031860 con 0x7ff014068920 2024-09-13T17:42:09.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.724+0000 7ff00b7fe640 1 -- 172.21.15.102:0/593086414 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff004031c20 con 0x7ff014068920 2024-09-13T17:42:09.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.724+0000 7ff01979d640 1 -- 172.21.15.102:0/593086414 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff004009390 con 0x7ff014068920 2024-09-13T17:42:09.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.724+0000 7ff01979d640 1 -- 172.21.15.102:0/593086414 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff01411c380 con 0x7ff014068920 2024-09-13T17:42:09.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.725+0000 7ff00b7fe640 1 -- 172.21.15.102:0/593086414 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7ff004039040 con 0x7ff014068920 2024-09-13T17:42:09.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.725+0000 7ff00b7fe640 1 -- 172.21.15.102:0/593086414 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7ff004062280 con 0x7ff014068920 2024-09-13T17:42:09.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.725+0000 7ff01979d640 1 -- 172.21.15.102:0/593086414 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff014064000 con 0x7ff014068920 2024-09-13T17:42:09.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.730+0000 7ff00b7fe640 1 -- 172.21.15.102:0/593086414 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7ff004037080 con 0x7ff014068920 2024-09-13T17:42:09.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.885+0000 7ff01979d640 1 -- 172.21.15.102:0/593086414 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff01411dc30 con 0x7ff014068920 2024-09-13T17:42:09.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.886+0000 7ff00b7fe640 1 -- 172.21.15.102:0/593086414 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+11375 (secure 0 0 0) 0x7ff004049020 con 0x7ff014068920 2024-09-13T17:42:09.888 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:42:09.888 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":20,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:42:07.375548+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:04.672265+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:05.583134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:09.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.888+0000 7ff01979d640 1 -- 172.21.15.102:0/593086414 >> v1:172.21.15.102:6816/34937 conn(0x7feff4031160 legacy=0x7feff4033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:09.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.888+0000 7ff01979d640 1 -- 172.21.15.102:0/593086414 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff014068920 msgr2=0x7ff01411b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:09.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.888+0000 7ff01979d640 1 --2- 172.21.15.102:0/593086414 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff014068920 0x7ff01411b020 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff00402a630 tx=0x7ff004032000 comp rx=0 tx=0).stop 2024-09-13T17:42:09.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.888+0000 7ff01979d640 1 -- 172.21.15.102:0/593086414 shutdown_connections 2024-09-13T17:42:09.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.888+0000 7ff01979d640 1 --2- 172.21.15.102:0/593086414 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff01412f310 0x7ff01413ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:09.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.888+0000 7ff01979d640 1 --2- 172.21.15.102:0/593086414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff01409dc50 0x7ff01411b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:09.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.888+0000 7ff01979d640 1 --2- 172.21.15.102:0/593086414 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff014068920 0x7ff01411b020 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:09.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.888+0000 7ff01979d640 1 -- 172.21.15.102:0/593086414 >> 172.21.15.102:0/593086414 conn(0x7ff01406f320 msgr2=0x7ff0140a1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:09.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.888+0000 7ff01979d640 1 -- 172.21.15.102:0/593086414 shutdown_connections 2024-09-13T17:42:09.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.888+0000 7ff01979d640 1 -- 172.21.15.102:0/593086414 wait complete. 2024-09-13T17:42:09.900 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-09-13T17:42:09.901 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:42:09.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.992+0000 7f747a709640 1 Processor -- start 2024-09-13T17:42:09.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.992+0000 7f747a709640 1 -- start start 2024-09-13T17:42:09.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.992+0000 7f747a709640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f747409abb0 0x7f747412c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:09.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.992+0000 7f747a709640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f747412c6a0 0x7f747412cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:09.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.992+0000 7f747a709640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f747406a530 con 0x7f7474096d80 2024-09-13T17:42:09.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.992+0000 7f747a709640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f747409af80 con 0x7f747409abb0 2024-09-13T17:42:09.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.992+0000 7f747a709640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f747412cff0 con 0x7f747412c6a0 2024-09-13T17:42:09.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.993+0000 7f7473fff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f747409abb0 0x7f747412c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:09.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.993+0000 7f74737fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f747412c6a0 0x7f747412cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:09.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.993+0000 7f7478ca6640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f7474096d80 0x7f74740971e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33592/0 (socket says 172.21.15.102:33592) 2024-09-13T17:42:09.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.993+0000 7f7478ca6640 1 -- 172.21.15.102:0/813854427 learned_addr learned my addr 172.21.15.102:0/813854427 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:42:09.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.993+0000 7f7473fff640 1 -- 172.21.15.102:0/813854427 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f747412c6a0 msgr2=0x7f747412cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:09.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.993+0000 7f7473fff640 1 --2- 172.21.15.102:0/813854427 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f747412c6a0 0x7f747412cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:09.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.993+0000 7f7473fff640 1 -- 172.21.15.102:0/813854427 >> v1:172.21.15.102:6789/0 conn(0x7f7474096d80 legacy=0x7f74740971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:09.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.993+0000 7f7473fff640 1 -- 172.21.15.102:0/813854427 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7474098be0 con 0x7f747409abb0 2024-09-13T17:42:09.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.993+0000 7f7472ffd640 1 -- 172.21.15.102:0/813854427 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3728078337 0 0) 0x7f747406a530 con 0x7f7474096d80 2024-09-13T17:42:09.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.993+0000 7f74737fe640 1 --2- 172.21.15.102:0/813854427 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f747412c6a0 0x7f747412cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:09.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.994+0000 7f7473fff640 1 --2- 172.21.15.102:0/813854427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f747409abb0 0x7f747412c160 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7464009880 tx=0x7f74640090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db787a70977ea947 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:10.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.994+0000 7f7472ffd640 1 -- 172.21.15.102:0/813854427 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746402ac50 con 0x7f747409abb0 2024-09-13T17:42:10.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.994+0000 7f7472ffd640 1 -- 172.21.15.102:0/813854427 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f746402adb0 con 0x7f747409abb0 2024-09-13T17:42:10.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.994+0000 7f7472ffd640 1 -- 172.21.15.102:0/813854427 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7464032670 con 0x7f747409abb0 2024-09-13T17:42:10.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.994+0000 7f747a709640 1 -- 172.21.15.102:0/813854427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f747409abb0 msgr2=0x7f747412c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:10.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.994+0000 7f747a709640 1 --2- 172.21.15.102:0/813854427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f747409abb0 0x7f747412c160 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7464009880 tx=0x7f74640090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.994+0000 7f747a709640 1 -- 172.21.15.102:0/813854427 shutdown_connections 2024-09-13T17:42:10.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.994+0000 7f747a709640 1 --2- 172.21.15.102:0/813854427 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f747412c6a0 0x7f747412cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.994+0000 7f747a709640 1 --2- 172.21.15.102:0/813854427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f747409abb0 0x7f747412c160 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.994+0000 7f747a709640 1 -- 172.21.15.102:0/813854427 >> 172.21.15.102:0/813854427 conn(0x7f747409e820 msgr2=0x7f747409ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:10.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.994+0000 7f747a709640 1 -- 172.21.15.102:0/813854427 shutdown_connections 2024-09-13T17:42:10.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.994+0000 7f747a709640 1 -- 172.21.15.102:0/813854427 wait complete. 2024-09-13T17:42:10.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.995+0000 7f747a709640 1 Processor -- start 2024-09-13T17:42:10.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.995+0000 7f747a709640 1 -- start start 2024-09-13T17:42:10.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.996+0000 7f747a709640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7474096d80 0x7f747413e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:10.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.996+0000 7f747a709640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f747409abb0 0x7f747413ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:10.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.996+0000 7f747a709640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f747412c6a0 0x7f747413f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:10.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.996+0000 7f747a709640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f747412a370 con 0x7f747409abb0 2024-09-13T17:42:10.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.996+0000 7f747a709640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74740981a0 con 0x7f747412c6a0 2024-09-13T17:42:10.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.996+0000 7f747a709640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7474116790 con 0x7f7474096d80 2024-09-13T17:42:10.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.996+0000 7f7473fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7474096d80 0x7f747413e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:10.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.996+0000 7f74737fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f747409abb0 0x7f747413ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:10.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.996+0000 7f7478ca6640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f747412c6a0 0x7f747413f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:10.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.996+0000 7f74737fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f747409abb0 0x7f747413ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52598/0 (socket says 172.21.15.102:52598) 2024-09-13T17:42:10.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.996+0000 7f7473fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7474096d80 0x7f747413e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:42058/0 (socket says 172.21.15.102:42058) 2024-09-13T17:42:10.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.996+0000 7f74737fe640 1 -- 172.21.15.102:0/2146532779 learned_addr learned my addr 172.21.15.102:0/2146532779 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:10.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.996+0000 7f74737fe640 1 -- 172.21.15.102:0/2146532779 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7474096d80 msgr2=0x7f747413e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:10.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.996+0000 7f74737fe640 1 --2- 172.21.15.102:0/2146532779 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7474096d80 0x7f747413e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.996+0000 7f74737fe640 1 -- 172.21.15.102:0/2146532779 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f747412c6a0 msgr2=0x7f747413f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:10.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.996+0000 7f74737fe640 1 --2- 172.21.15.102:0/2146532779 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f747412c6a0 0x7f747413f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.996+0000 7f74737fe640 1 -- 172.21.15.102:0/2146532779 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7468009390 con 0x7f747409abb0 2024-09-13T17:42:10.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.996+0000 7f7478ca6640 1 --2- 172.21.15.102:0/2146532779 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f747412c6a0 0x7f747413f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:10.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.996+0000 7f74737fe640 1 --2- 172.21.15.102:0/2146532779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f747409abb0 0x7f747413ed40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f746800ec80 tx=0x7f74680090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:10.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.996+0000 7f745bfff640 1 -- 172.21.15.102:0/2146532779 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746800eec0 con 0x7f747409abb0 2024-09-13T17:42:10.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.996+0000 7f745bfff640 1 -- 172.21.15.102:0/2146532779 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f746800cbc0 con 0x7f747409abb0 2024-09-13T17:42:10.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.996+0000 7f745bfff640 1 -- 172.21.15.102:0/2146532779 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7468013470 con 0x7f747409abb0 2024-09-13T17:42:10.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.996+0000 7f747a709640 1 -- 172.21.15.102:0/2146532779 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7464009390 con 0x7f747409abb0 2024-09-13T17:42:10.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.997+0000 7f747a709640 1 -- 172.21.15.102:0/2146532779 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f747409a4d0 con 0x7f747409abb0 2024-09-13T17:42:10.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.997+0000 7f745bfff640 1 -- 172.21.15.102:0/2146532779 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f74680137e0 con 0x7f747409abb0 2024-09-13T17:42:10.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.997+0000 7f745bfff640 1 -- 172.21.15.102:0/2146532779 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7f7468042860 con 0x7f747409abb0 2024-09-13T17:42:10.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:09.997+0000 7f747a709640 1 -- 172.21.15.102:0/2146532779 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7474098e00 con 0x7f747409abb0 2024-09-13T17:42:10.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.002+0000 7f745bfff640 1 -- 172.21.15.102:0/2146532779 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f746801bb90 con 0x7f747409abb0 2024-09-13T17:42:10.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.152+0000 7f747a709640 1 -- 172.21.15.102:0/2146532779 --> v1:172.21.15.102:6816/34937 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7474099e70 con 0x7f7460031160 2024-09-13T17:42:10.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.158+0000 7f745bfff640 1 -- 172.21.15.102:0/2146532779 <== mgr.4101 v1:172.21.15.102:6816/34937 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258729 (unknown 2965378022 0 1180813428) 0x7f7474099e70 con 0x7f7460031160 2024-09-13T17:42:10.163 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:42:10.164 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-09-13T17:42:10.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.161+0000 7f747a709640 1 -- 172.21.15.102:0/2146532779 >> v1:172.21.15.102:6816/34937 conn(0x7f7460031160 legacy=0x7f7460033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:10.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.161+0000 7f747a709640 1 -- 172.21.15.102:0/2146532779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f747409abb0 msgr2=0x7f747413ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:10.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.161+0000 7f747a709640 1 --2- 172.21.15.102:0/2146532779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f747409abb0 0x7f747413ed40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f746800ec80 tx=0x7f74680090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.161+0000 7f747a709640 1 -- 172.21.15.102:0/2146532779 shutdown_connections 2024-09-13T17:42:10.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.161+0000 7f747a709640 1 --2- 172.21.15.102:0/2146532779 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f747412c6a0 0x7f747413f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.161+0000 7f747a709640 1 --2- 172.21.15.102:0/2146532779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f747409abb0 0x7f747413ed40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.161+0000 7f747a709640 1 --2- 172.21.15.102:0/2146532779 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7474096d80 0x7f747413e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.161+0000 7f747a709640 1 -- 172.21.15.102:0/2146532779 >> 172.21.15.102:0/2146532779 conn(0x7f747409e820 msgr2=0x7f747412f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:10.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.161+0000 7f747a709640 1 -- 172.21.15.102:0/2146532779 shutdown_connections 2024-09-13T17:42:10.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.161+0000 7f747a709640 1 -- 172.21.15.102:0/2146532779 wait complete. 2024-09-13T17:42:10.173 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-13T17:42:08.412709+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1191148,"kb_used_data":1712,"kb_used_omap":6,"kb_used_meta":1189369,"kb_avail":376296212,"statfs":{"total":386547056640,"available":385327321088,"internally_reserved":0,"allocated":1753088,"data_stored":1335854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6816,"internal_metadata":1217914208},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000485"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:05.276096+0000","last_change":"2024-09-13T17:42:05.276096+0000","last_active":"2024-09-13T17:42:05.276096+0000","last_peered":"2024-09-13T17:42:05.276096+0000","last_clean":"2024-09-13T17:42:05.276096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:05.276096+0000","last_undegraded":"2024-09-13T17:42:05.276096+0000","last_fullsized":"2024-09-13T17:42:05.276096+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:05.276096+0000","last_change":"2024-09-13T17:42:05.276096+0000","last_active":"2024-09-13T17:42:05.276096+0000","last_peered":"2024-09-13T17:42:05.276096+0000","last_clean":"2024-09-13T17:42:05.276096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:05.276096+0000","last_undegraded":"2024-09-13T17:42:05.276096+0000","last_fullsized":"2024-09-13T17:42:05.276096+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:05.276096+0000","last_change":"2024-09-13T17:42:05.276096+0000","last_active":"2024-09-13T17:42:05.276096+0000","last_peered":"2024-09-13T17:42:05.276096+0000","last_clean":"2024-09-13T17:42:05.276096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:05.276096+0000","last_undegraded":"2024-09-13T17:42:05.276096+0000","last_fullsized":"2024-09-13T17:42:05.276096+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:05.276096+0000","last_change":"2024-09-13T17:42:05.276096+0000","last_active":"2024-09-13T17:42:05.276096+0000","last_peered":"2024-09-13T17:42:05.276096+0000","last_clean":"2024-09-13T17:42:05.276096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:05.276096+0000","last_undegraded":"2024-09-13T17:42:05.276096+0000","last_fullsized":"2024-09-13T17:42:05.276096+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:05.276096+0000","last_change":"2024-09-13T17:42:05.276096+0000","last_active":"2024-09-13T17:42:05.276096+0000","last_peered":"2024-09-13T17:42:05.276096+0000","last_clean":"2024-09-13T17:42:05.276096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:05.276096+0000","last_undegraded":"2024-09-13T17:42:05.276096+0000","last_fullsized":"2024-09-13T17:42:05.276096+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:05.276096+0000","last_change":"2024-09-13T17:42:05.276096+0000","last_active":"2024-09-13T17:42:05.276096+0000","last_peered":"2024-09-13T17:42:05.276096+0000","last_clean":"2024-09-13T17:42:05.276096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:05.276096+0000","last_undegraded":"2024-09-13T17:42:05.276096+0000","last_fullsized":"2024-09-13T17:42:05.276096+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:05.276096+0000","last_change":"2024-09-13T17:42:05.276096+0000","last_active":"2024-09-13T17:42:05.276096+0000","last_peered":"2024-09-13T17:42:05.276096+0000","last_clean":"2024-09-13T17:42:05.276096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:05.276096+0000","last_undegraded":"2024-09-13T17:42:05.276096+0000","last_fullsized":"2024-09-13T17:42:05.276096+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:05.276096+0000","last_change":"2024-09-13T17:42:05.276096+0000","last_active":"2024-09-13T17:42:05.276096+0000","last_peered":"2024-09-13T17:42:05.276096+0000","last_clean":"2024-09-13T17:42:05.276096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:05.276096+0000","last_undegraded":"2024-09-13T17:42:05.276096+0000","last_fullsized":"2024-09-13T17:42:05.276096+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:05.276096+0000","last_change":"2024-09-13T17:42:05.276096+0000","last_active":"2024-09-13T17:42:05.276096+0000","last_peered":"2024-09-13T17:42:05.276096+0000","last_clean":"2024-09-13T17:42:05.276096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:05.276096+0000","last_undegraded":"2024-09-13T17:42:05.276096+0000","last_fullsized":"2024-09-13T17:42:05.276096+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:05.276096+0000","last_change":"2024-09-13T17:42:05.276096+0000","last_active":"2024-09-13T17:42:05.276096+0000","last_peered":"2024-09-13T17:42:05.276096+0000","last_clean":"2024-09-13T17:42:05.276096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:05.276096+0000","last_undegraded":"2024-09-13T17:42:05.276096+0000","last_fullsized":"2024-09-13T17:42:05.276096+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:05.276096+0000","last_change":"2024-09-13T17:42:05.276096+0000","last_active":"2024-09-13T17:42:05.276096+0000","last_peered":"2024-09-13T17:42:05.276096+0000","last_clean":"2024-09-13T17:42:05.276096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:05.276096+0000","last_undegraded":"2024-09-13T17:42:05.276096+0000","last_fullsized":"2024-09-13T17:42:05.276096+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:05.276096+0000","last_change":"2024-09-13T17:42:05.276096+0000","last_active":"2024-09-13T17:42:05.276096+0000","last_peered":"2024-09-13T17:42:05.276096+0000","last_clean":"2024-09-13T17:42:05.276096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:05.276096+0000","last_undegraded":"2024-09-13T17:42:05.276096+0000","last_fullsized":"2024-09-13T17:42:05.276096+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":80,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:01.131711+0000","last_change":"2024-09-13T17:41:55.654539+0000","last_active":"2024-09-13T17:42:01.131711+0000","last_peered":"2024-09-13T17:42:01.131711+0000","last_clean":"2024-09-13T17:42:01.131711+0000","last_became_active":"2024-09-13T17:41:55.653536+0000","last_became_peered":"2024-09-13T17:41:55.653536+0000","last_unstale":"2024-09-13T17:42:01.131711+0000","last_undegraded":"2024-09-13T17:42:01.131711+0000","last_fullsized":"2024-09-13T17:42:01.131711+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:54.532965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:54.532965+0000","last_clean_scrub_stamp":"2024-09-13T17:41:54.532965+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:56.662588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":15,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-13T17:42:02.137183+0000","last_change":"2024-09-13T17:42:02.137356+0000","last_active":"2024-09-13T17:42:02.137183+0000","last_peered":"2024-09-13T17:42:02.137183+0000","last_clean":"2024-09-13T17:42:02.137183+0000","last_became_active":"2024-09-13T17:42:00.116534+0000","last_became_peered":"2024-09-13T17:42:00.116534+0000","last_unstale":"2024-09-13T17:42:02.137183+0000","last_undegraded":"2024-09-13T17:42:02.137183+0000","last_fullsized":"2024-09-13T17:42:02.137183+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:06.665067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000321799,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:05.276096+0000","last_change":"2024-09-13T17:42:05.276096+0000","last_active":"2024-09-13T17:42:05.276096+0000","last_peered":"2024-09-13T17:42:05.276096+0000","last_clean":"2024-09-13T17:42:05.276096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:05.276096+0000","last_undegraded":"2024-09-13T17:42:05.276096+0000","last_fullsized":"2024-09-13T17:42:05.276096+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-13T17:42:02.137146+0000","last_change":"2024-09-13T17:42:02.137373+0000","last_active":"2024-09-13T17:42:02.137146+0000","last_peered":"2024-09-13T17:42:02.137146+0000","last_clean":"2024-09-13T17:42:02.137146+0000","last_became_active":"2024-09-13T17:42:00.116542+0000","last_became_peered":"2024-09-13T17:42:00.116542+0000","last_unstale":"2024-09-13T17:42:02.137146+0000","last_undegraded":"2024-09-13T17:42:02.137146+0000","last_fullsized":"2024-09-13T17:42:02.137146+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:53.754563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035063,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:05.276096+0000","last_change":"2024-09-13T17:42:05.276096+0000","last_active":"2024-09-13T17:42:05.276096+0000","last_peered":"2024-09-13T17:42:05.276096+0000","last_clean":"2024-09-13T17:42:05.276096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:05.276096+0000","last_undegraded":"2024-09-13T17:42:05.276096+0000","last_fullsized":"2024-09-13T17:42:05.276096+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-13T17:42:02.615690+0000","last_change":"2024-09-13T17:42:02.615831+0000","last_active":"2024-09-13T17:42:02.615690+0000","last_peered":"2024-09-13T17:42:02.615690+0000","last_clean":"2024-09-13T17:42:02.615690+0000","last_became_active":"2024-09-13T17:42:00.116427+0000","last_became_peered":"2024-09-13T17:42:00.116427+0000","last_unstale":"2024-09-13T17:42:02.615690+0000","last_undegraded":"2024-09-13T17:42:02.615690+0000","last_fullsized":"2024-09-13T17:42:02.615690+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:29:17.848348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000228423,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:05.276096+0000","last_change":"2024-09-13T17:42:05.276096+0000","last_active":"2024-09-13T17:42:05.276096+0000","last_peered":"2024-09-13T17:42:05.276096+0000","last_clean":"2024-09-13T17:42:05.276096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:05.276096+0000","last_undegraded":"2024-09-13T17:42:05.276096+0000","last_fullsized":"2024-09-13T17:42:05.276096+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-13T17:42:02.138081+0000","last_change":"2024-09-13T17:42:02.138081+0000","last_active":"2024-09-13T17:42:02.138081+0000","last_peered":"2024-09-13T17:42:02.138081+0000","last_clean":"2024-09-13T17:42:02.138081+0000","last_became_active":"2024-09-13T17:42:00.116465+0000","last_became_peered":"2024-09-13T17:42:00.116465+0000","last_unstale":"2024-09-13T17:42:02.138081+0000","last_undegraded":"2024-09-13T17:42:02.138081+0000","last_fullsized":"2024-09-13T17:42:02.138081+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:40:30.342531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001026573,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:05.276096+0000","last_change":"2024-09-13T17:42:05.276096+0000","last_active":"2024-09-13T17:42:05.276096+0000","last_peered":"2024-09-13T17:42:05.276096+0000","last_clean":"2024-09-13T17:42:05.276096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:05.276096+0000","last_undegraded":"2024-09-13T17:42:05.276096+0000","last_fullsized":"2024-09-13T17:42:05.276096+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:05.276096+0000","last_change":"2024-09-13T17:42:05.276096+0000","last_active":"2024-09-13T17:42:05.276096+0000","last_peered":"2024-09-13T17:42:05.276096+0000","last_clean":"2024-09-13T17:42:05.276096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:05.276096+0000","last_undegraded":"2024-09-13T17:42:05.276096+0000","last_fullsized":"2024-09-13T17:42:05.276096+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-13T17:42:02.134689+0000","last_change":"2024-09-13T17:42:02.135135+0000","last_active":"2024-09-13T17:42:02.134689+0000","last_peered":"2024-09-13T17:42:02.134689+0000","last_clean":"2024-09-13T17:42:02.134689+0000","last_became_active":"2024-09-13T17:42:00.364240+0000","last_became_peered":"2024-09-13T17:42:00.364240+0000","last_unstale":"2024-09-13T17:42:02.134689+0000","last_undegraded":"2024-09-13T17:42:02.134689+0000","last_fullsized":"2024-09-13T17:42:02.134689+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:31:29.382006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575944,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:05.276096+0000","last_change":"2024-09-13T17:42:05.276096+0000","last_active":"2024-09-13T17:42:05.276096+0000","last_peered":"2024-09-13T17:42:05.276096+0000","last_clean":"2024-09-13T17:42:05.276096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:05.276096+0000","last_undegraded":"2024-09-13T17:42:05.276096+0000","last_fullsized":"2024-09-13T17:42:05.276096+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-13T17:42:02.137915+0000","last_change":"2024-09-13T17:42:02.138136+0000","last_active":"2024-09-13T17:42:02.137915+0000","last_peered":"2024-09-13T17:42:02.137915+0000","last_clean":"2024-09-13T17:42:02.137915+0000","last_became_active":"2024-09-13T17:42:00.364233+0000","last_became_peered":"2024-09-13T17:42:00.364233+0000","last_unstale":"2024-09-13T17:42:02.137915+0000","last_undegraded":"2024-09-13T17:42:02.137915+0000","last_fullsized":"2024-09-13T17:42:02.137915+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:54.234194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010757130000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:05.276096+0000","last_change":"2024-09-13T17:42:05.276096+0000","last_active":"2024-09-13T17:42:05.276096+0000","last_peered":"2024-09-13T17:42:05.276096+0000","last_clean":"2024-09-13T17:42:05.276096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:05.276096+0000","last_undegraded":"2024-09-13T17:42:05.276096+0000","last_fullsized":"2024-09-13T17:42:05.276096+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-13T17:42:02.134729+0000","last_change":"2024-09-13T17:42:02.134963+0000","last_active":"2024-09-13T17:42:02.134729+0000","last_peered":"2024-09-13T17:42:02.134729+0000","last_clean":"2024-09-13T17:42:02.134729+0000","last_became_active":"2024-09-13T17:42:00.364233+0000","last_became_peered":"2024-09-13T17:42:00.364233+0000","last_unstale":"2024-09-13T17:42:02.134729+0000","last_undegraded":"2024-09-13T17:42:02.134729+0000","last_fullsized":"2024-09-13T17:42:02.134729+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:24:31.191067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393223,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:05.276096+0000","last_change":"2024-09-13T17:42:05.276096+0000","last_active":"2024-09-13T17:42:05.276096+0000","last_peered":"2024-09-13T17:42:05.276096+0000","last_clean":"2024-09-13T17:42:05.276096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:05.276096+0000","last_undegraded":"2024-09-13T17:42:05.276096+0000","last_fullsized":"2024-09-13T17:42:05.276096+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-13T17:42:02.138137+0000","last_change":"2024-09-13T17:42:02.138362+0000","last_active":"2024-09-13T17:42:02.138137+0000","last_peered":"2024-09-13T17:42:02.138137+0000","last_clean":"2024-09-13T17:42:02.138137+0000","last_became_active":"2024-09-13T17:42:00.364214+0000","last_became_peered":"2024-09-13T17:42:00.364214+0000","last_unstale":"2024-09-13T17:42:02.138137+0000","last_undegraded":"2024-09-13T17:42:02.138137+0000","last_fullsized":"2024-09-13T17:42:02.138137+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:33.976642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012776910000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:05.276096+0000","last_change":"2024-09-13T17:42:05.276096+0000","last_active":"2024-09-13T17:42:05.276096+0000","last_peered":"2024-09-13T17:42:05.276096+0000","last_clean":"2024-09-13T17:42:05.276096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:05.276096+0000","last_undegraded":"2024-09-13T17:42:05.276096+0000","last_fullsized":"2024-09-13T17:42:05.276096+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:05.276096+0000","last_change":"2024-09-13T17:42:05.276096+0000","last_active":"2024-09-13T17:42:05.276096+0000","last_peered":"2024-09-13T17:42:05.276096+0000","last_clean":"2024-09-13T17:42:05.276096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:05.276096+0000","last_undegraded":"2024-09-13T17:42:05.276096+0000","last_fullsized":"2024-09-13T17:42:05.276096+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:05.276096+0000","last_change":"2024-09-13T17:42:05.276096+0000","last_active":"2024-09-13T17:42:05.276096+0000","last_peered":"2024-09-13T17:42:05.276096+0000","last_clean":"2024-09-13T17:42:05.276096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:05.276096+0000","last_undegraded":"2024-09-13T17:42:05.276096+0000","last_fullsized":"2024-09-13T17:42:05.276096+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:05.276096+0000","last_change":"2024-09-13T17:42:05.276096+0000","last_active":"2024-09-13T17:42:05.276096+0000","last_peered":"2024-09-13T17:42:05.276096+0000","last_clean":"2024-09-13T17:42:05.276096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:05.276096+0000","last_undegraded":"2024-09-13T17:42:05.276096+0000","last_fullsized":"2024-09-13T17:42:05.276096+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:05.276096+0000","last_change":"2024-09-13T17:42:05.276096+0000","last_active":"2024-09-13T17:42:05.276096+0000","last_peered":"2024-09-13T17:42:05.276096+0000","last_clean":"2024-09-13T17:42:05.276096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:05.276096+0000","last_undegraded":"2024-09-13T17:42:05.276096+0000","last_fullsized":"2024-09-13T17:42:05.276096+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:05.276096+0000","last_change":"2024-09-13T17:42:05.276096+0000","last_active":"2024-09-13T17:42:05.276096+0000","last_peered":"2024-09-13T17:42:05.276096+0000","last_clean":"2024-09-13T17:42:05.276096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:05.276096+0000","last_undegraded":"2024-09-13T17:42:05.276096+0000","last_fullsized":"2024-09-13T17:42:05.276096+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:05.276096+0000","last_change":"2024-09-13T17:42:05.276096+0000","last_active":"2024-09-13T17:42:05.276096+0000","last_peered":"2024-09-13T17:42:05.276096+0000","last_clean":"2024-09-13T17:42:05.276096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:05.276096+0000","last_undegraded":"2024-09-13T17:42:05.276096+0000","last_fullsized":"2024-09-13T17:42:05.276096+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:05.276096+0000","last_change":"2024-09-13T17:42:05.276096+0000","last_active":"2024-09-13T17:42:05.276096+0000","last_peered":"2024-09-13T17:42:05.276096+0000","last_clean":"2024-09-13T17:42:05.276096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:05.276096+0000","last_undegraded":"2024-09-13T17:42:05.276096+0000","last_fullsized":"2024-09-13T17:42:05.276096+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:05.276096+0000","last_change":"2024-09-13T17:42:05.276096+0000","last_active":"2024-09-13T17:42:05.276096+0000","last_peered":"2024-09-13T17:42:05.276096+0000","last_clean":"2024-09-13T17:42:05.276096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:05.276096+0000","last_undegraded":"2024-09-13T17:42:05.276096+0000","last_fullsized":"2024-09-13T17:42:05.276096+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:05.276096+0000","last_change":"2024-09-13T17:42:05.276096+0000","last_active":"2024-09-13T17:42:05.276096+0000","last_peered":"2024-09-13T17:42:05.276096+0000","last_clean":"2024-09-13T17:42:05.276096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:05.276096+0000","last_undegraded":"2024-09-13T17:42:05.276096+0000","last_fullsized":"2024-09-13T17:42:05.276096+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:05.276096+0000","last_change":"2024-09-13T17:42:05.276096+0000","last_active":"2024-09-13T17:42:05.276096+0000","last_peered":"2024-09-13T17:42:05.276096+0000","last_clean":"2024-09-13T17:42:05.276096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:05.276096+0000","last_undegraded":"2024-09-13T17:42:05.276096+0000","last_fullsized":"2024-09-13T17:42:05.276096+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:06.280075+0000","last_change":"2024-09-13T17:42:06.280075+0000","last_active":"2024-09-13T17:42:06.280075+0000","last_peered":"2024-09-13T17:42:06.280075+0000","last_clean":"2024-09-13T17:42:06.280075+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:06.280075+0000","last_undegraded":"2024-09-13T17:42:06.280075+0000","last_fullsized":"2024-09-13T17:42:06.280075+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:05.276096+0000","last_change":"2024-09-13T17:42:05.276096+0000","last_active":"2024-09-13T17:42:05.276096+0000","last_peered":"2024-09-13T17:42:05.276096+0000","last_clean":"2024-09-13T17:42:05.276096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:05.276096+0000","last_undegraded":"2024-09-13T17:42:05.276096+0000","last_fullsized":"2024-09-13T17:42:05.276096+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705667,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":143360,"data_stored":38789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]}]},{"osd":2,"up_from":9,"seq":38654705667,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297492,"kb_used_data":136,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074348,"statfs":{"total":96636764160,"available":96332132352,"internally_reserved":0,"allocated":139264,"data_stored":38770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28299999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]}]},{"osd":1,"up_from":9,"seq":38654705670,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298084,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073756,"statfs":{"total":96636764160,"available":96331526144,"internally_reserved":0,"allocated":737280,"data_stored":629157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23699999999999999}]}]},{"osd":0,"up_from":9,"seq":38654705668,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298072,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073768,"statfs":{"total":96636764160,"available":96331538432,"internally_reserved":0,"allocated":733184,"data_stored":629138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.19}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23300000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:42:10.178 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:42:10.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.304+0000 7f1ef68c9640 1 Processor -- start 2024-09-13T17:42:10.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.305+0000 7f1ef68c9640 1 -- start start 2024-09-13T17:42:10.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.305+0000 7f1ef68c9640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ef009abb0 0x7f1ef012c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:10.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.305+0000 7f1ef68c9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1ef012c6a0 0x7f1ef012cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:10.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.305+0000 7f1ef68c9640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ef006a530 con 0x7f1ef0096d80 2024-09-13T17:42:10.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.305+0000 7f1ef68c9640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef009af80 con 0x7f1ef009abb0 2024-09-13T17:42:10.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.305+0000 7f1ef68c9640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef012cff0 con 0x7f1ef012c6a0 2024-09-13T17:42:10.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.305+0000 7f1eef7fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1ef012c6a0 0x7f1ef012cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:10.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.305+0000 7f1ef4e66640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f1ef0096d80 0x7f1ef0097220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33602/0 (socket says 172.21.15.102:33602) 2024-09-13T17:42:10.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.305+0000 7f1eeffff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ef009abb0 0x7f1ef012c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:10.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.305+0000 7f1ef4e66640 1 -- 172.21.15.102:0/1067511319 learned_addr learned my addr 172.21.15.102:0/1067511319 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:42:10.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.305+0000 7f1eeeffd640 1 -- 172.21.15.102:0/1067511319 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 684191205 0 0) 0x7f1ef006a530 con 0x7f1ef0096d80 2024-09-13T17:42:10.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.305+0000 7f1eef7fe640 1 -- 172.21.15.102:0/1067511319 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ef009abb0 msgr2=0x7f1ef012c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:10.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.305+0000 7f1eef7fe640 1 --2- 172.21.15.102:0/1067511319 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ef009abb0 0x7f1ef012c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.305+0000 7f1eef7fe640 1 -- 172.21.15.102:0/1067511319 >> v1:172.21.15.102:6789/0 conn(0x7f1ef0096d80 legacy=0x7f1ef0097220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:10.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.305+0000 7f1eef7fe640 1 -- 172.21.15.102:0/1067511319 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef0098be0 con 0x7f1ef012c6a0 2024-09-13T17:42:10.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.305+0000 7f1eeffff640 1 --2- 172.21.15.102:0/1067511319 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ef009abb0 0x7f1ef012c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:10.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.306+0000 7f1eef7fe640 1 --2- 172.21.15.102:0/1067511319 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1ef012c6a0 0x7f1ef012cab0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8009880 tx=0x7f1ed80090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=630244fb47f60f5b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:10.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.306+0000 7f1eeeffd640 1 -- 172.21.15.102:0/1067511319 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed802ac50 con 0x7f1ef012c6a0 2024-09-13T17:42:10.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.306+0000 7f1eeeffd640 1 -- 172.21.15.102:0/1067511319 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed802adb0 con 0x7f1ef012c6a0 2024-09-13T17:42:10.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.306+0000 7f1eeeffd640 1 -- 172.21.15.102:0/1067511319 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed8032670 con 0x7f1ef012c6a0 2024-09-13T17:42:10.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.307+0000 7f1ef68c9640 1 -- 172.21.15.102:0/1067511319 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1ef012c6a0 msgr2=0x7f1ef012cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:10.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.307+0000 7f1ef68c9640 1 --2- 172.21.15.102:0/1067511319 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1ef012c6a0 0x7f1ef012cab0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8009880 tx=0x7f1ed80090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.307+0000 7f1ef68c9640 1 -- 172.21.15.102:0/1067511319 shutdown_connections 2024-09-13T17:42:10.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.307+0000 7f1ef68c9640 1 --2- 172.21.15.102:0/1067511319 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1ef012c6a0 0x7f1ef012cab0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.307+0000 7f1ef68c9640 1 --2- 172.21.15.102:0/1067511319 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ef009abb0 0x7f1ef012c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.307+0000 7f1ef68c9640 1 -- 172.21.15.102:0/1067511319 >> 172.21.15.102:0/1067511319 conn(0x7f1ef009e820 msgr2=0x7f1ef009ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:10.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.307+0000 7f1ef68c9640 1 -- 172.21.15.102:0/1067511319 shutdown_connections 2024-09-13T17:42:10.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.307+0000 7f1ef68c9640 1 -- 172.21.15.102:0/1067511319 wait complete. 2024-09-13T17:42:10.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.308+0000 7f1ef68c9640 1 Processor -- start 2024-09-13T17:42:10.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.308+0000 7f1ef68c9640 1 -- start start 2024-09-13T17:42:10.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.308+0000 7f1ef68c9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1ef0096d80 0x7f1ef01296b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:10.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.308+0000 7f1ef68c9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ef009abb0 0x7f1ef0117790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:10.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.308+0000 7f1ef68c9640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ef012c6a0 0x7f1ef0117cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:10.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.308+0000 7f1ef68c9640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef011b170 con 0x7f1ef009abb0 2024-09-13T17:42:10.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.308+0000 7f1ef68c9640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef01192d0 con 0x7f1ef012c6a0 2024-09-13T17:42:10.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.308+0000 7f1ef68c9640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef009d8f0 con 0x7f1ef0096d80 2024-09-13T17:42:10.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.308+0000 7f1eeffff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1ef0096d80 0x7f1ef01296b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:10.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.308+0000 7f1eef7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ef009abb0 0x7f1ef0117790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:10.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.308+0000 7f1eeffff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1ef0096d80 0x7f1ef01296b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:42078/0 (socket says 172.21.15.102:42078) 2024-09-13T17:42:10.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.308+0000 7f1eeffff640 1 -- 172.21.15.102:0/1253672345 learned_addr learned my addr 172.21.15.102:0/1253672345 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:10.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.308+0000 7f1ef4e66640 1 --2- 172.21.15.102:0/1253672345 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ef012c6a0 0x7f1ef0117cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:10.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.309+0000 7f1eeffff640 1 -- 172.21.15.102:0/1253672345 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ef012c6a0 msgr2=0x7f1ef0117cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:10.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.309+0000 7f1eeffff640 1 --2- 172.21.15.102:0/1253672345 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ef012c6a0 0x7f1ef0117cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.309+0000 7f1eeffff640 1 -- 172.21.15.102:0/1253672345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ef009abb0 msgr2=0x7f1ef0117790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:10.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.309+0000 7f1eeffff640 1 --2- 172.21.15.102:0/1253672345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ef009abb0 0x7f1ef0117790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.309+0000 7f1eeffff640 1 -- 172.21.15.102:0/1253672345 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee0009390 con 0x7f1ef0096d80 2024-09-13T17:42:10.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.309+0000 7f1eeffff640 1 --2- 172.21.15.102:0/1253672345 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1ef0096d80 0x7f1ef01296b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1ee000ec80 tx=0x7f1ee00090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:10.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.309+0000 7f1ee7fff640 1 -- 172.21.15.102:0/1253672345 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee000eec0 con 0x7f1ef0096d80 2024-09-13T17:42:10.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.309+0000 7f1ee7fff640 1 -- 172.21.15.102:0/1253672345 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee000cbc0 con 0x7f1ef0096d80 2024-09-13T17:42:10.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.309+0000 7f1ef68c9640 1 -- 172.21.15.102:0/1253672345 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ed8009390 con 0x7f1ef0096d80 2024-09-13T17:42:10.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.309+0000 7f1ee7fff640 1 -- 172.21.15.102:0/1253672345 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee0013470 con 0x7f1ef0096d80 2024-09-13T17:42:10.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.309+0000 7f1ef68c9640 1 -- 172.21.15.102:0/1253672345 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef011b830 con 0x7f1ef0096d80 2024-09-13T17:42:10.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.310+0000 7f1ee7fff640 1 -- 172.21.15.102:0/1253672345 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f1ee00137e0 con 0x7f1ef0096d80 2024-09-13T17:42:10.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.310+0000 7f1ee7fff640 1 -- 172.21.15.102:0/1253672345 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7f1ee0042480 con 0x7f1ef0096d80 2024-09-13T17:42:10.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.310+0000 7f1ef68c9640 1 -- 172.21.15.102:0/1253672345 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ef0098e00 con 0x7f1ef0096d80 2024-09-13T17:42:10.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.314+0000 7f1ee7fff640 1 -- 172.21.15.102:0/1253672345 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f1ee0021070 con 0x7f1ef0096d80 2024-09-13T17:42:10.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.471+0000 7f1ef68c9640 1 -- 172.21.15.102:0/1253672345 --> v1:172.21.15.102:6816/34937 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1ef0064000 con 0x7f1ed0031190 2024-09-13T17:42:10.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.477+0000 7f1ee7fff640 1 -- 172.21.15.102:0/1253672345 <== mgr.4101 v1:172.21.15.102:6816/34937 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+271444 (unknown 2965378022 0 24283526) 0x7f1ef0064000 con 0x7f1ed0031190 2024-09-13T17:42:10.482 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:42:10.483 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-09-13T17:42:10.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.480+0000 7f1ef68c9640 1 -- 172.21.15.102:0/1253672345 >> v1:172.21.15.102:6816/34937 conn(0x7f1ed0031190 legacy=0x7f1ed0033640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:10.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.480+0000 7f1ef68c9640 1 -- 172.21.15.102:0/1253672345 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1ef0096d80 msgr2=0x7f1ef01296b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:10.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.480+0000 7f1ef68c9640 1 --2- 172.21.15.102:0/1253672345 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1ef0096d80 0x7f1ef01296b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1ee000ec80 tx=0x7f1ee00090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.480+0000 7f1ef68c9640 1 -- 172.21.15.102:0/1253672345 shutdown_connections 2024-09-13T17:42:10.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.480+0000 7f1ef68c9640 1 --2- 172.21.15.102:0/1253672345 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1ef012c6a0 0x7f1ef0117cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.480+0000 7f1ef68c9640 1 --2- 172.21.15.102:0/1253672345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ef009abb0 0x7f1ef0117790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.480+0000 7f1ef68c9640 1 --2- 172.21.15.102:0/1253672345 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1ef0096d80 0x7f1ef01296b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.480+0000 7f1ef68c9640 1 -- 172.21.15.102:0/1253672345 >> 172.21.15.102:0/1253672345 conn(0x7f1ef009e820 msgr2=0x7f1ef006a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:10.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.480+0000 7f1ef68c9640 1 -- 172.21.15.102:0/1253672345 shutdown_connections 2024-09-13T17:42:10.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.480+0000 7f1ef68c9640 1 -- 172.21.15.102:0/1253672345 wait complete. 2024-09-13T17:42:10.492 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-13T17:42:10.413000+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592725,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":125,"num_write_kb":1121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":93,"ondisk_log_size":93,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1191328,"kb_used_data":1888,"kb_used_omap":6,"kb_used_meta":1189369,"kb_avail":376296032,"statfs":{"total":386547056640,"available":385327136768,"internally_reserved":0,"allocated":1933312,"data_stored":1399992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6816,"internal_metadata":1217914208},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.131370"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.334754+0000","last_change":"2024-09-13T17:42:07.332154+0000","last_active":"2024-09-13T17:42:07.334754+0000","last_peered":"2024-09-13T17:42:07.334754+0000","last_clean":"2024-09-13T17:42:07.334754+0000","last_became_active":"2024-09-13T17:42:07.331874+0000","last_became_peered":"2024-09-13T17:42:07.331874+0000","last_unstale":"2024-09-13T17:42:07.334754+0000","last_undegraded":"2024-09-13T17:42:07.334754+0000","last_fullsized":"2024-09-13T17:42:07.334754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:10:22.943375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.329161+0000","last_change":"2024-09-13T17:42:07.326086+0000","last_active":"2024-09-13T17:42:07.329161+0000","last_peered":"2024-09-13T17:42:07.329161+0000","last_clean":"2024-09-13T17:42:07.329161+0000","last_became_active":"2024-09-13T17:42:07.325830+0000","last_became_peered":"2024-09-13T17:42:07.325830+0000","last_unstale":"2024-09-13T17:42:07.329161+0000","last_undegraded":"2024-09-13T17:42:07.329161+0000","last_fullsized":"2024-09-13T17:42:07.329161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:54.719512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338441+0000","last_change":"2024-09-13T17:42:07.326953+0000","last_active":"2024-09-13T17:42:07.338441+0000","last_peered":"2024-09-13T17:42:07.338441+0000","last_clean":"2024-09-13T17:42:07.338441+0000","last_became_active":"2024-09-13T17:42:07.326717+0000","last_became_peered":"2024-09-13T17:42:07.326717+0000","last_unstale":"2024-09-13T17:42:07.338441+0000","last_undegraded":"2024-09-13T17:42:07.338441+0000","last_fullsized":"2024-09-13T17:42:07.338441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:20.647404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338424+0000","last_change":"2024-09-13T17:42:07.326356+0000","last_active":"2024-09-13T17:42:07.338424+0000","last_peered":"2024-09-13T17:42:07.338424+0000","last_clean":"2024-09-13T17:42:07.338424+0000","last_became_active":"2024-09-13T17:42:07.326127+0000","last_became_peered":"2024-09-13T17:42:07.326127+0000","last_unstale":"2024-09-13T17:42:07.338424+0000","last_undegraded":"2024-09-13T17:42:07.338424+0000","last_fullsized":"2024-09-13T17:42:07.338424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:44.405426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338304+0000","last_change":"2024-09-13T17:42:07.330092+0000","last_active":"2024-09-13T17:42:07.338304+0000","last_peered":"2024-09-13T17:42:07.338304+0000","last_clean":"2024-09-13T17:42:07.338304+0000","last_became_active":"2024-09-13T17:42:07.327658+0000","last_became_peered":"2024-09-13T17:42:07.327658+0000","last_unstale":"2024-09-13T17:42:07.338304+0000","last_undegraded":"2024-09-13T17:42:07.338304+0000","last_fullsized":"2024-09-13T17:42:07.338304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:24:02.920430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.324569+0000","last_change":"2024-09-13T17:42:07.303609+0000","last_active":"2024-09-13T17:42:07.324569+0000","last_peered":"2024-09-13T17:42:07.324569+0000","last_clean":"2024-09-13T17:42:07.324569+0000","last_became_active":"2024-09-13T17:42:07.303384+0000","last_became_peered":"2024-09-13T17:42:07.303384+0000","last_unstale":"2024-09-13T17:42:07.324569+0000","last_undegraded":"2024-09-13T17:42:07.324569+0000","last_fullsized":"2024-09-13T17:42:07.324569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:43:56.000592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.326264+0000","last_change":"2024-09-13T17:42:07.307630+0000","last_active":"2024-09-13T17:42:07.326264+0000","last_peered":"2024-09-13T17:42:07.326264+0000","last_clean":"2024-09-13T17:42:07.326264+0000","last_became_active":"2024-09-13T17:42:07.307382+0000","last_became_peered":"2024-09-13T17:42:07.307382+0000","last_unstale":"2024-09-13T17:42:07.326264+0000","last_undegraded":"2024-09-13T17:42:07.326264+0000","last_fullsized":"2024-09-13T17:42:07.326264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:26:32.846287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.326676+0000","last_change":"2024-09-13T17:42:07.320736+0000","last_active":"2024-09-13T17:42:07.326676+0000","last_peered":"2024-09-13T17:42:07.326676+0000","last_clean":"2024-09-13T17:42:07.326676+0000","last_became_active":"2024-09-13T17:42:07.319801+0000","last_became_peered":"2024-09-13T17:42:07.319801+0000","last_unstale":"2024-09-13T17:42:07.326676+0000","last_undegraded":"2024-09-13T17:42:07.326676+0000","last_fullsized":"2024-09-13T17:42:07.326676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:28:32.030397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.339799+0000","last_change":"2024-09-13T17:42:07.335208+0000","last_active":"2024-09-13T17:42:07.339799+0000","last_peered":"2024-09-13T17:42:07.339799+0000","last_clean":"2024-09-13T17:42:07.339799+0000","last_became_active":"2024-09-13T17:42:07.334956+0000","last_became_peered":"2024-09-13T17:42:07.334956+0000","last_unstale":"2024-09-13T17:42:07.339799+0000","last_undegraded":"2024-09-13T17:42:07.339799+0000","last_fullsized":"2024-09-13T17:42:07.339799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:59:53.544086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.325383+0000","last_change":"2024-09-13T17:42:07.317057+0000","last_active":"2024-09-13T17:42:07.325383+0000","last_peered":"2024-09-13T17:42:07.325383+0000","last_clean":"2024-09-13T17:42:07.325383+0000","last_became_active":"2024-09-13T17:42:07.316429+0000","last_became_peered":"2024-09-13T17:42:07.316429+0000","last_unstale":"2024-09-13T17:42:07.325383+0000","last_undegraded":"2024-09-13T17:42:07.325383+0000","last_fullsized":"2024-09-13T17:42:07.325383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:59:53.544086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.325590+0000","last_change":"2024-09-13T17:42:07.321106+0000","last_active":"2024-09-13T17:42:07.325590+0000","last_peered":"2024-09-13T17:42:07.325590+0000","last_clean":"2024-09-13T17:42:07.325590+0000","last_became_active":"2024-09-13T17:42:07.320827+0000","last_became_peered":"2024-09-13T17:42:07.320827+0000","last_unstale":"2024-09-13T17:42:07.325590+0000","last_undegraded":"2024-09-13T17:42:07.325590+0000","last_fullsized":"2024-09-13T17:42:07.325590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:34:34.794230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.334785+0000","last_change":"2024-09-13T17:42:07.331865+0000","last_active":"2024-09-13T17:42:07.334785+0000","last_peered":"2024-09-13T17:42:07.334785+0000","last_clean":"2024-09-13T17:42:07.334785+0000","last_became_active":"2024-09-13T17:42:07.331419+0000","last_became_peered":"2024-09-13T17:42:07.331419+0000","last_unstale":"2024-09-13T17:42:07.334785+0000","last_undegraded":"2024-09-13T17:42:07.334785+0000","last_fullsized":"2024-09-13T17:42:07.334785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:48.155823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.335811+0000","last_change":"2024-09-13T17:42:07.333296+0000","last_active":"2024-09-13T17:42:07.335811+0000","last_peered":"2024-09-13T17:42:07.335811+0000","last_clean":"2024-09-13T17:42:07.335811+0000","last_became_active":"2024-09-13T17:42:07.332898+0000","last_became_peered":"2024-09-13T17:42:07.332898+0000","last_unstale":"2024-09-13T17:42:07.335811+0000","last_undegraded":"2024-09-13T17:42:07.335811+0000","last_fullsized":"2024-09-13T17:42:07.335811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:20.241542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338928+0000","last_change":"2024-09-13T17:42:07.324771+0000","last_active":"2024-09-13T17:42:07.338928+0000","last_peered":"2024-09-13T17:42:07.338928+0000","last_clean":"2024-09-13T17:42:07.338928+0000","last_became_active":"2024-09-13T17:42:07.323820+0000","last_became_peered":"2024-09-13T17:42:07.323820+0000","last_unstale":"2024-09-13T17:42:07.338928+0000","last_undegraded":"2024-09-13T17:42:07.338928+0000","last_fullsized":"2024-09-13T17:42:07.338928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:24:02.920430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.337730+0000","last_change":"2024-09-13T17:42:07.325827+0000","last_active":"2024-09-13T17:42:07.337730+0000","last_peered":"2024-09-13T17:42:07.337730+0000","last_clean":"2024-09-13T17:42:07.337730+0000","last_became_active":"2024-09-13T17:42:07.325589+0000","last_became_peered":"2024-09-13T17:42:07.325589+0000","last_unstale":"2024-09-13T17:42:07.337730+0000","last_undegraded":"2024-09-13T17:42:07.337730+0000","last_fullsized":"2024-09-13T17:42:07.337730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:43.466051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.334093+0000","last_change":"2024-09-13T17:42:07.315478+0000","last_active":"2024-09-13T17:42:07.334093+0000","last_peered":"2024-09-13T17:42:07.334093+0000","last_clean":"2024-09-13T17:42:07.334093+0000","last_became_active":"2024-09-13T17:42:07.315218+0000","last_became_peered":"2024-09-13T17:42:07.315218+0000","last_unstale":"2024-09-13T17:42:07.334093+0000","last_undegraded":"2024-09-13T17:42:07.334093+0000","last_fullsized":"2024-09-13T17:42:07.334093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:06:44.563276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338279+0000","last_change":"2024-09-13T17:42:07.328192+0000","last_active":"2024-09-13T17:42:07.338279+0000","last_peered":"2024-09-13T17:42:07.338279+0000","last_clean":"2024-09-13T17:42:07.338279+0000","last_became_active":"2024-09-13T17:42:07.327951+0000","last_became_peered":"2024-09-13T17:42:07.327951+0000","last_unstale":"2024-09-13T17:42:07.338279+0000","last_undegraded":"2024-09-13T17:42:07.338279+0000","last_fullsized":"2024-09-13T17:42:07.338279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:28:32.030397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.320769+0000","last_change":"2024-09-13T17:42:07.314816+0000","last_active":"2024-09-13T17:42:07.320769+0000","last_peered":"2024-09-13T17:42:07.320769+0000","last_clean":"2024-09-13T17:42:07.320769+0000","last_became_active":"2024-09-13T17:42:07.314464+0000","last_became_peered":"2024-09-13T17:42:07.314464+0000","last_unstale":"2024-09-13T17:42:07.320769+0000","last_undegraded":"2024-09-13T17:42:07.320769+0000","last_fullsized":"2024-09-13T17:42:07.320769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:17.483066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.335387+0000","last_change":"2024-09-13T17:42:07.332269+0000","last_active":"2024-09-13T17:42:07.335387+0000","last_peered":"2024-09-13T17:42:07.335387+0000","last_clean":"2024-09-13T17:42:07.335387+0000","last_became_active":"2024-09-13T17:42:07.332040+0000","last_became_peered":"2024-09-13T17:42:07.332040+0000","last_unstale":"2024-09-13T17:42:07.335387+0000","last_undegraded":"2024-09-13T17:42:07.335387+0000","last_fullsized":"2024-09-13T17:42:07.335387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:48:49.368747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.340627+0000","last_change":"2024-09-13T17:42:07.335425+0000","last_active":"2024-09-13T17:42:07.340627+0000","last_peered":"2024-09-13T17:42:07.340627+0000","last_clean":"2024-09-13T17:42:07.340627+0000","last_became_active":"2024-09-13T17:42:07.335173+0000","last_became_peered":"2024-09-13T17:42:07.335173+0000","last_unstale":"2024-09-13T17:42:07.340627+0000","last_undegraded":"2024-09-13T17:42:07.340627+0000","last_fullsized":"2024-09-13T17:42:07.340627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:00:10.355609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.340514+0000","last_change":"2024-09-13T17:42:07.331183+0000","last_active":"2024-09-13T17:42:07.340514+0000","last_peered":"2024-09-13T17:42:07.340514+0000","last_clean":"2024-09-13T17:42:07.340514+0000","last_became_active":"2024-09-13T17:42:07.330936+0000","last_became_peered":"2024-09-13T17:42:07.330936+0000","last_unstale":"2024-09-13T17:42:07.340514+0000","last_undegraded":"2024-09-13T17:42:07.340514+0000","last_fullsized":"2024-09-13T17:42:07.340514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:42:38.505262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.337694+0000","last_change":"2024-09-13T17:42:07.325877+0000","last_active":"2024-09-13T17:42:07.337694+0000","last_peered":"2024-09-13T17:42:07.337694+0000","last_clean":"2024-09-13T17:42:07.337694+0000","last_became_active":"2024-09-13T17:42:07.325657+0000","last_became_peered":"2024-09-13T17:42:07.325657+0000","last_unstale":"2024-09-13T17:42:07.337694+0000","last_undegraded":"2024-09-13T17:42:07.337694+0000","last_fullsized":"2024-09-13T17:42:07.337694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:48.663311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331637+0000","last_change":"2024-09-13T17:42:07.322059+0000","last_active":"2024-09-13T17:42:07.331637+0000","last_peered":"2024-09-13T17:42:07.331637+0000","last_clean":"2024-09-13T17:42:07.331637+0000","last_became_active":"2024-09-13T17:42:07.312536+0000","last_became_peered":"2024-09-13T17:42:07.312536+0000","last_unstale":"2024-09-13T17:42:07.331637+0000","last_undegraded":"2024-09-13T17:42:07.331637+0000","last_fullsized":"2024-09-13T17:42:07.331637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:14:08.780968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.340774+0000","last_change":"2024-09-13T17:42:07.336557+0000","last_active":"2024-09-13T17:42:07.340774+0000","last_peered":"2024-09-13T17:42:07.340774+0000","last_clean":"2024-09-13T17:42:07.340774+0000","last_became_active":"2024-09-13T17:42:07.335820+0000","last_became_peered":"2024-09-13T17:42:07.335820+0000","last_unstale":"2024-09-13T17:42:07.340774+0000","last_undegraded":"2024-09-13T17:42:07.340774+0000","last_fullsized":"2024-09-13T17:42:07.340774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:20.647404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.342633+0000","last_change":"2024-09-13T17:42:07.337695+0000","last_active":"2024-09-13T17:42:07.342633+0000","last_peered":"2024-09-13T17:42:07.342633+0000","last_clean":"2024-09-13T17:42:07.342633+0000","last_became_active":"2024-09-13T17:42:07.337419+0000","last_became_peered":"2024-09-13T17:42:07.337419+0000","last_unstale":"2024-09-13T17:42:07.342633+0000","last_undegraded":"2024-09-13T17:42:07.342633+0000","last_fullsized":"2024-09-13T17:42:07.342633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:22:40.880345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.325365+0000","last_change":"2024-09-13T17:42:07.315008+0000","last_active":"2024-09-13T17:42:07.325365+0000","last_peered":"2024-09-13T17:42:07.325365+0000","last_clean":"2024-09-13T17:42:07.325365+0000","last_became_active":"2024-09-13T17:42:07.313477+0000","last_became_peered":"2024-09-13T17:42:07.313477+0000","last_unstale":"2024-09-13T17:42:07.325365+0000","last_undegraded":"2024-09-13T17:42:07.325365+0000","last_fullsized":"2024-09-13T17:42:07.325365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:25.632002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331694+0000","last_change":"2024-09-13T17:42:07.317949+0000","last_active":"2024-09-13T17:42:07.331694+0000","last_peered":"2024-09-13T17:42:07.331694+0000","last_clean":"2024-09-13T17:42:07.331694+0000","last_became_active":"2024-09-13T17:42:07.317720+0000","last_became_peered":"2024-09-13T17:42:07.317720+0000","last_unstale":"2024-09-13T17:42:07.331694+0000","last_undegraded":"2024-09-13T17:42:07.331694+0000","last_fullsized":"2024-09-13T17:42:07.331694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:31.378417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.324990+0000","last_change":"2024-09-13T17:42:07.313528+0000","last_active":"2024-09-13T17:42:07.324990+0000","last_peered":"2024-09-13T17:42:07.324990+0000","last_clean":"2024-09-13T17:42:07.324990+0000","last_became_active":"2024-09-13T17:42:07.313092+0000","last_became_peered":"2024-09-13T17:42:07.313092+0000","last_unstale":"2024-09-13T17:42:07.324990+0000","last_undegraded":"2024-09-13T17:42:07.324990+0000","last_fullsized":"2024-09-13T17:42:07.324990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:48:49.368747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331153+0000","last_change":"2024-09-13T17:42:07.314816+0000","last_active":"2024-09-13T17:42:07.331153+0000","last_peered":"2024-09-13T17:42:07.331153+0000","last_clean":"2024-09-13T17:42:07.331153+0000","last_became_active":"2024-09-13T17:42:07.314574+0000","last_became_peered":"2024-09-13T17:42:07.314574+0000","last_unstale":"2024-09-13T17:42:07.331153+0000","last_undegraded":"2024-09-13T17:42:07.331153+0000","last_fullsized":"2024-09-13T17:42:07.331153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:03:35.976183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338349+0000","last_change":"2024-09-13T17:42:07.328522+0000","last_active":"2024-09-13T17:42:07.338349+0000","last_peered":"2024-09-13T17:42:07.338349+0000","last_clean":"2024-09-13T17:42:07.338349+0000","last_became_active":"2024-09-13T17:42:07.327893+0000","last_became_peered":"2024-09-13T17:42:07.327893+0000","last_unstale":"2024-09-13T17:42:07.338349+0000","last_undegraded":"2024-09-13T17:42:07.338349+0000","last_fullsized":"2024-09-13T17:42:07.338349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:02:50.060149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331620+0000","last_change":"2024-09-13T17:42:07.321900+0000","last_active":"2024-09-13T17:42:07.331620+0000","last_peered":"2024-09-13T17:42:07.331620+0000","last_clean":"2024-09-13T17:42:07.331620+0000","last_became_active":"2024-09-13T17:42:07.319230+0000","last_became_peered":"2024-09-13T17:42:07.319230+0000","last_unstale":"2024-09-13T17:42:07.331620+0000","last_undegraded":"2024-09-13T17:42:07.331620+0000","last_fullsized":"2024-09-13T17:42:07.331620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:50:46.442172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.326655+0000","last_change":"2024-09-13T17:42:07.315846+0000","last_active":"2024-09-13T17:42:07.326655+0000","last_peered":"2024-09-13T17:42:07.326655+0000","last_clean":"2024-09-13T17:42:07.326655+0000","last_became_active":"2024-09-13T17:42:07.315513+0000","last_became_peered":"2024-09-13T17:42:07.315513+0000","last_unstale":"2024-09-13T17:42:07.326655+0000","last_undegraded":"2024-09-13T17:42:07.326655+0000","last_fullsized":"2024-09-13T17:42:07.326655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:56:21.343829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331104+0000","last_change":"2024-09-13T17:42:07.315400+0000","last_active":"2024-09-13T17:42:07.331104+0000","last_peered":"2024-09-13T17:42:07.331104+0000","last_clean":"2024-09-13T17:42:07.331104+0000","last_became_active":"2024-09-13T17:42:07.315131+0000","last_became_peered":"2024-09-13T17:42:07.315131+0000","last_unstale":"2024-09-13T17:42:07.331104+0000","last_undegraded":"2024-09-13T17:42:07.331104+0000","last_fullsized":"2024-09-13T17:42:07.331104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:21:39.335847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338775+0000","last_change":"2024-09-13T17:42:06.308504+0000","last_active":"2024-09-13T17:42:07.338775+0000","last_peered":"2024-09-13T17:42:07.338775+0000","last_clean":"2024-09-13T17:42:07.338775+0000","last_became_active":"2024-09-13T17:42:06.308199+0000","last_became_peered":"2024-09-13T17:42:06.308199+0000","last_unstale":"2024-09-13T17:42:07.338775+0000","last_undegraded":"2024-09-13T17:42:07.338775+0000","last_fullsized":"2024-09-13T17:42:07.338775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:05.930342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331918+0000","last_change":"2024-09-13T17:42:07.322597+0000","last_active":"2024-09-13T17:42:07.331918+0000","last_peered":"2024-09-13T17:42:07.331918+0000","last_clean":"2024-09-13T17:42:07.331918+0000","last_became_active":"2024-09-13T17:42:07.321945+0000","last_became_peered":"2024-09-13T17:42:07.321945+0000","last_unstale":"2024-09-13T17:42:07.331918+0000","last_undegraded":"2024-09-13T17:42:07.331918+0000","last_fullsized":"2024-09-13T17:42:07.331918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:03:28.132876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.301168+0000","last_change":"2024-09-13T17:42:06.306367+0000","last_active":"2024-09-13T17:42:07.301168+0000","last_peered":"2024-09-13T17:42:07.301168+0000","last_clean":"2024-09-13T17:42:07.301168+0000","last_became_active":"2024-09-13T17:42:06.306146+0000","last_became_peered":"2024-09-13T17:42:06.306146+0000","last_unstale":"2024-09-13T17:42:07.301168+0000","last_undegraded":"2024-09-13T17:42:07.301168+0000","last_fullsized":"2024-09-13T17:42:07.301168+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:45:01.750995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338414+0000","last_change":"2024-09-13T17:42:07.329528+0000","last_active":"2024-09-13T17:42:07.338414+0000","last_peered":"2024-09-13T17:42:07.338414+0000","last_clean":"2024-09-13T17:42:07.338414+0000","last_became_active":"2024-09-13T17:42:07.329296+0000","last_became_peered":"2024-09-13T17:42:07.329296+0000","last_unstale":"2024-09-13T17:42:07.338414+0000","last_undegraded":"2024-09-13T17:42:07.338414+0000","last_fullsized":"2024-09-13T17:42:07.338414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:33:57.543704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.294808+0000","last_change":"2024-09-13T17:42:06.312798+0000","last_active":"2024-09-13T17:42:07.294808+0000","last_peered":"2024-09-13T17:42:07.294808+0000","last_clean":"2024-09-13T17:42:07.294808+0000","last_became_active":"2024-09-13T17:42:06.312388+0000","last_became_peered":"2024-09-13T17:42:06.312388+0000","last_unstale":"2024-09-13T17:42:07.294808+0000","last_undegraded":"2024-09-13T17:42:07.294808+0000","last_fullsized":"2024-09-13T17:42:07.294808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:56:20.339850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331165+0000","last_change":"2024-09-13T17:42:07.315290+0000","last_active":"2024-09-13T17:42:07.331165+0000","last_peered":"2024-09-13T17:42:07.331165+0000","last_clean":"2024-09-13T17:42:07.331165+0000","last_became_active":"2024-09-13T17:42:07.314670+0000","last_became_peered":"2024-09-13T17:42:07.314670+0000","last_unstale":"2024-09-13T17:42:07.331165+0000","last_undegraded":"2024-09-13T17:42:07.331165+0000","last_fullsized":"2024-09-13T17:42:07.331165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:00:24.969812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.340570+0000","last_change":"2024-09-13T17:42:06.305806+0000","last_active":"2024-09-13T17:42:07.340570+0000","last_peered":"2024-09-13T17:42:07.340570+0000","last_clean":"2024-09-13T17:42:07.340570+0000","last_became_active":"2024-09-13T17:42:06.305576+0000","last_became_peered":"2024-09-13T17:42:06.305576+0000","last_unstale":"2024-09-13T17:42:07.340570+0000","last_undegraded":"2024-09-13T17:42:07.340570+0000","last_fullsized":"2024-09-13T17:42:07.340570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:47:16.711770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.337092+0000","last_change":"2024-09-13T17:42:06.304106+0000","last_active":"2024-09-13T17:42:07.337092+0000","last_peered":"2024-09-13T17:42:07.337092+0000","last_clean":"2024-09-13T17:42:07.337092+0000","last_became_active":"2024-09-13T17:42:06.303845+0000","last_became_peered":"2024-09-13T17:42:06.303845+0000","last_unstale":"2024-09-13T17:42:07.337092+0000","last_undegraded":"2024-09-13T17:42:07.337092+0000","last_fullsized":"2024-09-13T17:42:07.337092+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:36:02.512243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.334525+0000","last_change":"2024-09-13T17:42:07.331147+0000","last_active":"2024-09-13T17:42:07.334525+0000","last_peered":"2024-09-13T17:42:07.334525+0000","last_clean":"2024-09-13T17:42:07.334525+0000","last_became_active":"2024-09-13T17:42:07.329386+0000","last_became_peered":"2024-09-13T17:42:07.329386+0000","last_unstale":"2024-09-13T17:42:07.334525+0000","last_undegraded":"2024-09-13T17:42:07.334525+0000","last_fullsized":"2024-09-13T17:42:07.334525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:59:21.210970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.321593+0000","last_change":"2024-09-13T17:42:06.304872+0000","last_active":"2024-09-13T17:42:07.321593+0000","last_peered":"2024-09-13T17:42:07.321593+0000","last_clean":"2024-09-13T17:42:07.321593+0000","last_became_active":"2024-09-13T17:42:06.304639+0000","last_became_peered":"2024-09-13T17:42:06.304639+0000","last_unstale":"2024-09-13T17:42:07.321593+0000","last_undegraded":"2024-09-13T17:42:07.321593+0000","last_fullsized":"2024-09-13T17:42:07.321593+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:40.147410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.340505+0000","last_change":"2024-09-13T17:42:07.331235+0000","last_active":"2024-09-13T17:42:07.340505+0000","last_peered":"2024-09-13T17:42:07.340505+0000","last_clean":"2024-09-13T17:42:07.340505+0000","last_became_active":"2024-09-13T17:42:07.330822+0000","last_became_peered":"2024-09-13T17:42:07.330822+0000","last_unstale":"2024-09-13T17:42:07.340505+0000","last_undegraded":"2024-09-13T17:42:07.340505+0000","last_fullsized":"2024-09-13T17:42:07.340505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:38:04.013092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.340781+0000","last_change":"2024-09-13T17:42:06.310319+0000","last_active":"2024-09-13T17:42:07.340781+0000","last_peered":"2024-09-13T17:42:07.340781+0000","last_clean":"2024-09-13T17:42:07.340781+0000","last_became_active":"2024-09-13T17:42:06.310085+0000","last_became_peered":"2024-09-13T17:42:06.310085+0000","last_unstale":"2024-09-13T17:42:07.340781+0000","last_undegraded":"2024-09-13T17:42:07.340781+0000","last_fullsized":"2024-09-13T17:42:07.340781+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:53.715533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.335838+0000","last_change":"2024-09-13T17:42:07.333488+0000","last_active":"2024-09-13T17:42:07.335838+0000","last_peered":"2024-09-13T17:42:07.335838+0000","last_clean":"2024-09-13T17:42:07.335838+0000","last_became_active":"2024-09-13T17:42:07.332200+0000","last_became_peered":"2024-09-13T17:42:07.332200+0000","last_unstale":"2024-09-13T17:42:07.335838+0000","last_undegraded":"2024-09-13T17:42:07.335838+0000","last_fullsized":"2024-09-13T17:42:07.335838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:52.666716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.303967+0000","last_change":"2024-09-13T17:42:06.308854+0000","last_active":"2024-09-13T17:42:07.303967+0000","last_peered":"2024-09-13T17:42:07.303967+0000","last_clean":"2024-09-13T17:42:07.303967+0000","last_became_active":"2024-09-13T17:42:06.307173+0000","last_became_peered":"2024-09-13T17:42:06.307173+0000","last_unstale":"2024-09-13T17:42:07.303967+0000","last_undegraded":"2024-09-13T17:42:07.303967+0000","last_fullsized":"2024-09-13T17:42:07.303967+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:59.621392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338229+0000","last_change":"2024-09-13T17:42:07.327431+0000","last_active":"2024-09-13T17:42:07.338229+0000","last_peered":"2024-09-13T17:42:07.338229+0000","last_clean":"2024-09-13T17:42:07.338229+0000","last_became_active":"2024-09-13T17:42:07.327202+0000","last_became_peered":"2024-09-13T17:42:07.327202+0000","last_unstale":"2024-09-13T17:42:07.338229+0000","last_undegraded":"2024-09-13T17:42:07.338229+0000","last_fullsized":"2024-09-13T17:42:07.338229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:42:38.505262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338801+0000","last_change":"2024-09-13T17:42:06.308795+0000","last_active":"2024-09-13T17:42:07.338801+0000","last_peered":"2024-09-13T17:42:07.338801+0000","last_clean":"2024-09-13T17:42:07.338801+0000","last_became_active":"2024-09-13T17:42:06.308557+0000","last_became_peered":"2024-09-13T17:42:06.308557+0000","last_unstale":"2024-09-13T17:42:07.338801+0000","last_undegraded":"2024-09-13T17:42:07.338801+0000","last_fullsized":"2024-09-13T17:42:07.338801+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:51.662738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338442+0000","last_change":"2024-09-13T17:42:07.327849+0000","last_active":"2024-09-13T17:42:07.338442+0000","last_peered":"2024-09-13T17:42:07.338442+0000","last_clean":"2024-09-13T17:42:07.338442+0000","last_became_active":"2024-09-13T17:42:07.327578+0000","last_became_peered":"2024-09-13T17:42:07.327578+0000","last_unstale":"2024-09-13T17:42:07.338442+0000","last_undegraded":"2024-09-13T17:42:07.338442+0000","last_fullsized":"2024-09-13T17:42:07.338442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:34:34.794230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.294395+0000","last_change":"2024-09-13T17:42:06.310708+0000","last_active":"2024-09-13T17:42:07.294395+0000","last_peered":"2024-09-13T17:42:07.294395+0000","last_clean":"2024-09-13T17:42:07.294395+0000","last_became_active":"2024-09-13T17:42:06.310476+0000","last_became_peered":"2024-09-13T17:42:06.310476+0000","last_unstale":"2024-09-13T17:42:07.294395+0000","last_undegraded":"2024-09-13T17:42:07.294395+0000","last_fullsized":"2024-09-13T17:42:07.294395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:10:21.939396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331677+0000","last_change":"2024-09-13T17:42:07.317665+0000","last_active":"2024-09-13T17:42:07.331677+0000","last_peered":"2024-09-13T17:42:07.331677+0000","last_clean":"2024-09-13T17:42:07.331677+0000","last_became_active":"2024-09-13T17:42:07.317401+0000","last_became_peered":"2024-09-13T17:42:07.317401+0000","last_unstale":"2024-09-13T17:42:07.331677+0000","last_undegraded":"2024-09-13T17:42:07.331677+0000","last_fullsized":"2024-09-13T17:42:07.331677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:12:55.460668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.295483+0000","last_change":"2024-09-13T17:42:06.313488+0000","last_active":"2024-09-13T17:42:07.295483+0000","last_peered":"2024-09-13T17:42:07.295483+0000","last_clean":"2024-09-13T17:42:07.295483+0000","last_became_active":"2024-09-13T17:42:06.313241+0000","last_became_peered":"2024-09-13T17:42:06.313241+0000","last_unstale":"2024-09-13T17:42:07.295483+0000","last_undegraded":"2024-09-13T17:42:07.295483+0000","last_fullsized":"2024-09-13T17:42:07.295483+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:48:48.364768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.335898+0000","last_change":"2024-09-13T17:42:07.333605+0000","last_active":"2024-09-13T17:42:07.335898+0000","last_peered":"2024-09-13T17:42:07.335898+0000","last_clean":"2024-09-13T17:42:07.335898+0000","last_became_active":"2024-09-13T17:42:07.332509+0000","last_became_peered":"2024-09-13T17:42:07.332509+0000","last_unstale":"2024-09-13T17:42:07.335898+0000","last_undegraded":"2024-09-13T17:42:07.335898+0000","last_fullsized":"2024-09-13T17:42:07.335898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:26:11.888585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.295399+0000","last_change":"2024-09-13T17:42:06.310629+0000","last_active":"2024-09-13T17:42:07.295399+0000","last_peered":"2024-09-13T17:42:07.295399+0000","last_clean":"2024-09-13T17:42:07.295399+0000","last_became_active":"2024-09-13T17:42:06.310103+0000","last_became_peered":"2024-09-13T17:42:06.310103+0000","last_unstale":"2024-09-13T17:42:07.295399+0000","last_undegraded":"2024-09-13T17:42:07.295399+0000","last_fullsized":"2024-09-13T17:42:07.295399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:42.121791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.340677+0000","last_change":"2024-09-13T17:42:07.335957+0000","last_active":"2024-09-13T17:42:07.340677+0000","last_peered":"2024-09-13T17:42:07.340677+0000","last_clean":"2024-09-13T17:42:07.340677+0000","last_became_active":"2024-09-13T17:42:07.331187+0000","last_became_peered":"2024-09-13T17:42:07.331187+0000","last_unstale":"2024-09-13T17:42:07.340677+0000","last_undegraded":"2024-09-13T17:42:07.340677+0000","last_fullsized":"2024-09-13T17:42:07.340677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:44.405426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":84,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.308327+0000","last_change":"2024-09-13T17:41:55.654539+0000","last_active":"2024-09-13T17:42:07.308327+0000","last_peered":"2024-09-13T17:42:07.308327+0000","last_clean":"2024-09-13T17:42:07.308327+0000","last_became_active":"2024-09-13T17:41:55.653536+0000","last_became_peered":"2024-09-13T17:41:55.653536+0000","last_unstale":"2024-09-13T17:42:07.308327+0000","last_undegraded":"2024-09-13T17:42:07.308327+0000","last_fullsized":"2024-09-13T17:42:07.308327+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:54.532965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:54.532965+0000","last_clean_scrub_stamp":"2024-09-13T17:41:54.532965+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:56.662588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.825620+0000","last_change":"2024-09-13T17:42:02.137356+0000","last_active":"2024-09-13T17:42:07.825620+0000","last_peered":"2024-09-13T17:42:07.825620+0000","last_clean":"2024-09-13T17:42:07.825620+0000","last_became_active":"2024-09-13T17:42:00.116534+0000","last_became_peered":"2024-09-13T17:42:00.116534+0000","last_unstale":"2024-09-13T17:42:07.825620+0000","last_undegraded":"2024-09-13T17:42:07.825620+0000","last_fullsized":"2024-09-13T17:42:07.825620+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:06.665067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000321799,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.294511+0000","last_change":"2024-09-13T17:42:06.312144+0000","last_active":"2024-09-13T17:42:07.294511+0000","last_peered":"2024-09-13T17:42:07.294511+0000","last_clean":"2024-09-13T17:42:07.294511+0000","last_became_active":"2024-09-13T17:42:06.311843+0000","last_became_peered":"2024-09-13T17:42:06.311843+0000","last_unstale":"2024-09-13T17:42:07.294511+0000","last_undegraded":"2024-09-13T17:42:07.294511+0000","last_fullsized":"2024-09-13T17:42:07.294511+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:16.479088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.332628+0000","last_change":"2024-09-13T17:42:07.329208+0000","last_active":"2024-09-13T17:42:07.332628+0000","last_peered":"2024-09-13T17:42:07.332628+0000","last_clean":"2024-09-13T17:42:07.332628+0000","last_became_active":"2024-09-13T17:42:07.328947+0000","last_became_peered":"2024-09-13T17:42:07.328947+0000","last_unstale":"2024-09-13T17:42:07.332628+0000","last_undegraded":"2024-09-13T17:42:07.332628+0000","last_fullsized":"2024-09-13T17:42:07.332628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:25:39.337142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.824598+0000","last_change":"2024-09-13T17:42:02.137373+0000","last_active":"2024-09-13T17:42:07.824598+0000","last_peered":"2024-09-13T17:42:07.824598+0000","last_clean":"2024-09-13T17:42:07.824598+0000","last_became_active":"2024-09-13T17:42:00.116542+0000","last_became_peered":"2024-09-13T17:42:00.116542+0000","last_unstale":"2024-09-13T17:42:07.824598+0000","last_undegraded":"2024-09-13T17:42:07.824598+0000","last_fullsized":"2024-09-13T17:42:07.824598+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:53.754563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035063,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.303137+0000","last_change":"2024-09-13T17:42:06.304286+0000","last_active":"2024-09-13T17:42:07.303137+0000","last_peered":"2024-09-13T17:42:07.303137+0000","last_clean":"2024-09-13T17:42:07.303137+0000","last_became_active":"2024-09-13T17:42:06.304053+0000","last_became_peered":"2024-09-13T17:42:06.304053+0000","last_unstale":"2024-09-13T17:42:07.303137+0000","last_undegraded":"2024-09-13T17:42:07.303137+0000","last_fullsized":"2024-09-13T17:42:07.303137+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:20:52.415835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338397+0000","last_change":"2024-09-13T17:42:07.326878+0000","last_active":"2024-09-13T17:42:07.338397+0000","last_peered":"2024-09-13T17:42:07.338397+0000","last_clean":"2024-09-13T17:42:07.338397+0000","last_became_active":"2024-09-13T17:42:07.326416+0000","last_became_peered":"2024-09-13T17:42:07.326416+0000","last_unstale":"2024-09-13T17:42:07.338397+0000","last_undegraded":"2024-09-13T17:42:07.338397+0000","last_fullsized":"2024-09-13T17:42:07.338397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:45.068616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.384334+0000","last_change":"2024-09-13T17:42:02.615831+0000","last_active":"2024-09-13T17:42:07.384334+0000","last_peered":"2024-09-13T17:42:07.384334+0000","last_clean":"2024-09-13T17:42:07.384334+0000","last_became_active":"2024-09-13T17:42:00.116427+0000","last_became_peered":"2024-09-13T17:42:00.116427+0000","last_unstale":"2024-09-13T17:42:07.384334+0000","last_undegraded":"2024-09-13T17:42:07.384334+0000","last_fullsized":"2024-09-13T17:42:07.384334+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:29:17.848348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000228423,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.310191+0000","last_change":"2024-09-13T17:42:06.302978+0000","last_active":"2024-09-13T17:42:07.310191+0000","last_peered":"2024-09-13T17:42:07.310191+0000","last_clean":"2024-09-13T17:42:07.310191+0000","last_became_active":"2024-09-13T17:42:06.302485+0000","last_became_peered":"2024-09-13T17:42:06.302485+0000","last_unstale":"2024-09-13T17:42:07.310191+0000","last_undegraded":"2024-09-13T17:42:07.310191+0000","last_fullsized":"2024-09-13T17:42:07.310191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:17:23.133953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.325491+0000","last_change":"2024-09-13T17:42:07.318090+0000","last_active":"2024-09-13T17:42:07.325491+0000","last_peered":"2024-09-13T17:42:07.325491+0000","last_clean":"2024-09-13T17:42:07.325491+0000","last_became_active":"2024-09-13T17:42:07.317337+0000","last_became_peered":"2024-09-13T17:42:07.317337+0000","last_unstale":"2024-09-13T17:42:07.325491+0000","last_undegraded":"2024-09-13T17:42:07.325491+0000","last_fullsized":"2024-09-13T17:42:07.325491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:02:50.060149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.824778+0000","last_change":"2024-09-13T17:42:02.138081+0000","last_active":"2024-09-13T17:42:07.824778+0000","last_peered":"2024-09-13T17:42:07.824778+0000","last_clean":"2024-09-13T17:42:07.824778+0000","last_became_active":"2024-09-13T17:42:00.116465+0000","last_became_peered":"2024-09-13T17:42:00.116465+0000","last_unstale":"2024-09-13T17:42:07.824778+0000","last_undegraded":"2024-09-13T17:42:07.824778+0000","last_fullsized":"2024-09-13T17:42:07.824778+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:40:30.342531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001026573,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.308940+0000","last_change":"2024-09-13T17:42:06.304017+0000","last_active":"2024-09-13T17:42:07.308940+0000","last_peered":"2024-09-13T17:42:07.308940+0000","last_clean":"2024-09-13T17:42:07.308940+0000","last_became_active":"2024-09-13T17:42:06.303764+0000","last_became_peered":"2024-09-13T17:42:06.303764+0000","last_unstale":"2024-09-13T17:42:07.308940+0000","last_undegraded":"2024-09-13T17:42:07.308940+0000","last_fullsized":"2024-09-13T17:42:07.308940+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:20:51.407014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.325010+0000","last_change":"2024-09-13T17:42:07.314438+0000","last_active":"2024-09-13T17:42:07.325010+0000","last_peered":"2024-09-13T17:42:07.325010+0000","last_clean":"2024-09-13T17:42:07.325010+0000","last_became_active":"2024-09-13T17:42:07.314196+0000","last_became_peered":"2024-09-13T17:42:07.314196+0000","last_unstale":"2024-09-13T17:42:07.325010+0000","last_undegraded":"2024-09-13T17:42:07.325010+0000","last_fullsized":"2024-09-13T17:42:07.325010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:33:57.543704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.298253+0000","last_change":"2024-09-13T17:42:06.304819+0000","last_active":"2024-09-13T17:42:07.298253+0000","last_peered":"2024-09-13T17:42:07.298253+0000","last_clean":"2024-09-13T17:42:07.298253+0000","last_became_active":"2024-09-13T17:42:06.304542+0000","last_became_peered":"2024-09-13T17:42:06.304542+0000","last_unstale":"2024-09-13T17:42:07.298253+0000","last_undegraded":"2024-09-13T17:42:07.298253+0000","last_fullsized":"2024-09-13T17:42:07.298253+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:10:57.127180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T17:42:08.701451+0000","last_change":"2024-09-13T17:42:02.135135+0000","last_active":"2024-09-13T17:42:08.701451+0000","last_peered":"2024-09-13T17:42:08.701451+0000","last_clean":"2024-09-13T17:42:08.701451+0000","last_became_active":"2024-09-13T17:42:00.364240+0000","last_became_peered":"2024-09-13T17:42:00.364240+0000","last_unstale":"2024-09-13T17:42:08.701451+0000","last_undegraded":"2024-09-13T17:42:08.701451+0000","last_fullsized":"2024-09-13T17:42:08.701451+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:31:29.382006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575944,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.342655+0000","last_change":"2024-09-13T17:42:07.338051+0000","last_active":"2024-09-13T17:42:07.342655+0000","last_peered":"2024-09-13T17:42:07.342655+0000","last_clean":"2024-09-13T17:42:07.342655+0000","last_became_active":"2024-09-13T17:42:07.337334+0000","last_became_peered":"2024-09-13T17:42:07.337334+0000","last_unstale":"2024-09-13T17:42:07.342655+0000","last_undegraded":"2024-09-13T17:42:07.342655+0000","last_fullsized":"2024-09-13T17:42:07.342655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:45.068616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.302622+0000","last_change":"2024-09-13T17:42:06.300680+0000","last_active":"2024-09-13T17:42:07.302622+0000","last_peered":"2024-09-13T17:42:07.302622+0000","last_clean":"2024-09-13T17:42:07.302622+0000","last_became_active":"2024-09-13T17:42:06.299518+0000","last_became_peered":"2024-09-13T17:42:06.299518+0000","last_unstale":"2024-09-13T17:42:07.302622+0000","last_undegraded":"2024-09-13T17:42:07.302622+0000","last_fullsized":"2024-09-13T17:42:07.302622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:20:52.415835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.825916+0000","last_change":"2024-09-13T17:42:02.138136+0000","last_active":"2024-09-13T17:42:07.825916+0000","last_peered":"2024-09-13T17:42:07.825916+0000","last_clean":"2024-09-13T17:42:07.825916+0000","last_became_active":"2024-09-13T17:42:00.364233+0000","last_became_peered":"2024-09-13T17:42:00.364233+0000","last_unstale":"2024-09-13T17:42:07.825916+0000","last_undegraded":"2024-09-13T17:42:07.825916+0000","last_fullsized":"2024-09-13T17:42:07.825916+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:54.234194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010757130000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.322084+0000","last_change":"2024-09-13T17:42:07.315025+0000","last_active":"2024-09-13T17:42:07.322084+0000","last_peered":"2024-09-13T17:42:07.322084+0000","last_clean":"2024-09-13T17:42:07.322084+0000","last_became_active":"2024-09-13T17:42:07.314074+0000","last_became_peered":"2024-09-13T17:42:07.314074+0000","last_unstale":"2024-09-13T17:42:07.322084+0000","last_undegraded":"2024-09-13T17:42:07.322084+0000","last_fullsized":"2024-09-13T17:42:07.322084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:25.632002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.340850+0000","last_change":"2024-09-13T17:42:06.310475+0000","last_active":"2024-09-13T17:42:07.340850+0000","last_peered":"2024-09-13T17:42:07.340850+0000","last_clean":"2024-09-13T17:42:07.340850+0000","last_became_active":"2024-09-13T17:42:06.310230+0000","last_became_peered":"2024-09-13T17:42:06.310230+0000","last_unstale":"2024-09-13T17:42:07.340850+0000","last_undegraded":"2024-09-13T17:42:07.340850+0000","last_fullsized":"2024-09-13T17:42:07.340850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:30:29.499136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T17:42:08.701455+0000","last_change":"2024-09-13T17:42:02.134963+0000","last_active":"2024-09-13T17:42:08.701455+0000","last_peered":"2024-09-13T17:42:08.701455+0000","last_clean":"2024-09-13T17:42:08.701455+0000","last_became_active":"2024-09-13T17:42:00.364233+0000","last_became_peered":"2024-09-13T17:42:00.364233+0000","last_unstale":"2024-09-13T17:42:08.701455+0000","last_undegraded":"2024-09-13T17:42:08.701455+0000","last_fullsized":"2024-09-13T17:42:08.701455+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:24:31.191067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393223,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.334115+0000","last_change":"2024-09-13T17:42:07.324146+0000","last_active":"2024-09-13T17:42:07.334115+0000","last_peered":"2024-09-13T17:42:07.334115+0000","last_clean":"2024-09-13T17:42:07.334115+0000","last_became_active":"2024-09-13T17:42:07.323120+0000","last_became_peered":"2024-09-13T17:42:07.323120+0000","last_unstale":"2024-09-13T17:42:07.334115+0000","last_undegraded":"2024-09-13T17:42:07.334115+0000","last_fullsized":"2024-09-13T17:42:07.334115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:20:26.946513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.310176+0000","last_change":"2024-09-13T17:42:06.305416+0000","last_active":"2024-09-13T17:42:07.310176+0000","last_peered":"2024-09-13T17:42:07.310176+0000","last_clean":"2024-09-13T17:42:07.310176+0000","last_became_active":"2024-09-13T17:42:06.304782+0000","last_became_peered":"2024-09-13T17:42:06.304782+0000","last_unstale":"2024-09-13T17:42:07.310176+0000","last_undegraded":"2024-09-13T17:42:07.310176+0000","last_fullsized":"2024-09-13T17:42:07.310176+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:08:01.276478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.825679+0000","last_change":"2024-09-13T17:42:02.138362+0000","last_active":"2024-09-13T17:42:07.825679+0000","last_peered":"2024-09-13T17:42:07.825679+0000","last_clean":"2024-09-13T17:42:07.825679+0000","last_became_active":"2024-09-13T17:42:00.364214+0000","last_became_peered":"2024-09-13T17:42:00.364214+0000","last_unstale":"2024-09-13T17:42:07.825679+0000","last_undegraded":"2024-09-13T17:42:07.825679+0000","last_fullsized":"2024-09-13T17:42:07.825679+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:33.976642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012776910000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.334811+0000","last_change":"2024-09-13T17:42:07.332265+0000","last_active":"2024-09-13T17:42:07.334811+0000","last_peered":"2024-09-13T17:42:07.334811+0000","last_clean":"2024-09-13T17:42:07.334811+0000","last_became_active":"2024-09-13T17:42:07.331834+0000","last_became_peered":"2024-09-13T17:42:07.331834+0000","last_unstale":"2024-09-13T17:42:07.334811+0000","last_undegraded":"2024-09-13T17:42:07.334811+0000","last_fullsized":"2024-09-13T17:42:07.334811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:52:45.662321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.335827+0000","last_change":"2024-09-13T17:42:07.333367+0000","last_active":"2024-09-13T17:42:07.335827+0000","last_peered":"2024-09-13T17:42:07.335827+0000","last_clean":"2024-09-13T17:42:07.335827+0000","last_became_active":"2024-09-13T17:42:07.333132+0000","last_became_peered":"2024-09-13T17:42:07.333132+0000","last_unstale":"2024-09-13T17:42:07.335827+0000","last_undegraded":"2024-09-13T17:42:07.335827+0000","last_fullsized":"2024-09-13T17:42:07.335827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:43.125769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"19'6","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.342572+0000","last_change":"2024-09-13T17:42:06.305919+0000","last_active":"2024-09-13T17:42:07.342572+0000","last_peered":"2024-09-13T17:42:07.342572+0000","last_clean":"2024-09-13T17:42:07.342572+0000","last_became_active":"2024-09-13T17:42:06.305663+0000","last_became_peered":"2024-09-13T17:42:06.305663+0000","last_unstale":"2024-09-13T17:42:07.342572+0000","last_undegraded":"2024-09-13T17:42:07.342572+0000","last_fullsized":"2024-09-13T17:42:07.342572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:24.628023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.340404+0000","last_change":"2024-09-13T17:42:07.330596+0000","last_active":"2024-09-13T17:42:07.340404+0000","last_peered":"2024-09-13T17:42:07.340404+0000","last_clean":"2024-09-13T17:42:07.340404+0000","last_became_active":"2024-09-13T17:42:07.330349+0000","last_became_peered":"2024-09-13T17:42:07.330349+0000","last_unstale":"2024-09-13T17:42:07.340404+0000","last_undegraded":"2024-09-13T17:42:07.340404+0000","last_fullsized":"2024-09-13T17:42:07.340404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:43.466051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.334688+0000","last_change":"2024-09-13T17:42:06.309276+0000","last_active":"2024-09-13T17:42:07.334688+0000","last_peered":"2024-09-13T17:42:07.334688+0000","last_clean":"2024-09-13T17:42:07.334688+0000","last_became_active":"2024-09-13T17:42:06.307311+0000","last_became_peered":"2024-09-13T17:42:06.307311+0000","last_unstale":"2024-09-13T17:42:07.334688+0000","last_undegraded":"2024-09-13T17:42:07.334688+0000","last_fullsized":"2024-09-13T17:42:07.334688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:31:06.878681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.299687+0000","last_change":"2024-09-13T17:42:06.301289+0000","last_active":"2024-09-13T17:42:07.299687+0000","last_peered":"2024-09-13T17:42:07.299687+0000","last_clean":"2024-09-13T17:42:07.299687+0000","last_became_active":"2024-09-13T17:42:06.301040+0000","last_became_peered":"2024-09-13T17:42:06.301040+0000","last_unstale":"2024-09-13T17:42:07.299687+0000","last_undegraded":"2024-09-13T17:42:07.299687+0000","last_fullsized":"2024-09-13T17:42:07.299687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:49:37.251198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331603+0000","last_change":"2024-09-13T17:42:07.321515+0000","last_active":"2024-09-13T17:42:07.331603+0000","last_peered":"2024-09-13T17:42:07.331603+0000","last_clean":"2024-09-13T17:42:07.331603+0000","last_became_active":"2024-09-13T17:42:07.312719+0000","last_became_peered":"2024-09-13T17:42:07.312719+0000","last_unstale":"2024-09-13T17:42:07.331603+0000","last_undegraded":"2024-09-13T17:42:07.331603+0000","last_fullsized":"2024-09-13T17:42:07.331603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:29:48.289765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.308212+0000","last_change":"2024-09-13T17:42:06.307317+0000","last_active":"2024-09-13T17:42:07.308212+0000","last_peered":"2024-09-13T17:42:07.308212+0000","last_clean":"2024-09-13T17:42:07.308212+0000","last_became_active":"2024-09-13T17:42:06.307087+0000","last_became_peered":"2024-09-13T17:42:06.307087+0000","last_unstale":"2024-09-13T17:42:07.308212+0000","last_undegraded":"2024-09-13T17:42:07.308212+0000","last_fullsized":"2024-09-13T17:42:07.308212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:47:16.711770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.339794+0000","last_change":"2024-09-13T17:42:07.333996+0000","last_active":"2024-09-13T17:42:07.339794+0000","last_peered":"2024-09-13T17:42:07.339794+0000","last_clean":"2024-09-13T17:42:07.339794+0000","last_became_active":"2024-09-13T17:42:07.329707+0000","last_became_peered":"2024-09-13T17:42:07.329707+0000","last_unstale":"2024-09-13T17:42:07.339794+0000","last_undegraded":"2024-09-13T17:42:07.339794+0000","last_fullsized":"2024-09-13T17:42:07.339794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:01:57.881608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331148+0000","last_change":"2024-09-13T17:42:07.314437+0000","last_active":"2024-09-13T17:42:07.331148+0000","last_peered":"2024-09-13T17:42:07.331148+0000","last_clean":"2024-09-13T17:42:07.331148+0000","last_became_active":"2024-09-13T17:42:07.314194+0000","last_became_peered":"2024-09-13T17:42:07.314194+0000","last_unstale":"2024-09-13T17:42:07.331148+0000","last_undegraded":"2024-09-13T17:42:07.331148+0000","last_fullsized":"2024-09-13T17:42:07.331148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:42:37.705801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.295292+0000","last_change":"2024-09-13T17:42:06.308176+0000","last_active":"2024-09-13T17:42:07.295292+0000","last_peered":"2024-09-13T17:42:07.295292+0000","last_clean":"2024-09-13T17:42:07.295292+0000","last_became_active":"2024-09-13T17:42:06.307906+0000","last_became_peered":"2024-09-13T17:42:06.307906+0000","last_unstale":"2024-09-13T17:42:07.295292+0000","last_undegraded":"2024-09-13T17:42:07.295292+0000","last_fullsized":"2024-09-13T17:42:07.295292+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:26:10.884606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338381+0000","last_change":"2024-09-13T17:42:07.326636+0000","last_active":"2024-09-13T17:42:07.338381+0000","last_peered":"2024-09-13T17:42:07.338381+0000","last_clean":"2024-09-13T17:42:07.338381+0000","last_became_active":"2024-09-13T17:42:07.326146+0000","last_became_peered":"2024-09-13T17:42:07.326146+0000","last_unstale":"2024-09-13T17:42:07.338381+0000","last_undegraded":"2024-09-13T17:42:07.338381+0000","last_fullsized":"2024-09-13T17:42:07.338381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:22:40.880345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.298760+0000","last_change":"2024-09-13T17:42:06.312966+0000","last_active":"2024-09-13T17:42:07.298760+0000","last_peered":"2024-09-13T17:42:07.298760+0000","last_clean":"2024-09-13T17:42:07.298760+0000","last_became_active":"2024-09-13T17:42:06.312726+0000","last_became_peered":"2024-09-13T17:42:06.312726+0000","last_unstale":"2024-09-13T17:42:07.298760+0000","last_undegraded":"2024-09-13T17:42:07.298760+0000","last_fullsized":"2024-09-13T17:42:07.298760+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:59.621392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331661+0000","last_change":"2024-09-13T17:42:07.315788+0000","last_active":"2024-09-13T17:42:07.331661+0000","last_peered":"2024-09-13T17:42:07.331661+0000","last_clean":"2024-09-13T17:42:07.331661+0000","last_became_active":"2024-09-13T17:42:07.312751+0000","last_became_peered":"2024-09-13T17:42:07.312751+0000","last_unstale":"2024-09-13T17:42:07.331661+0000","last_undegraded":"2024-09-13T17:42:07.331661+0000","last_fullsized":"2024-09-13T17:42:07.331661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:24:19.756257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.326108+0000","last_change":"2024-09-13T17:42:06.303521+0000","last_active":"2024-09-13T17:42:07.326108+0000","last_peered":"2024-09-13T17:42:07.326108+0000","last_clean":"2024-09-13T17:42:07.326108+0000","last_became_active":"2024-09-13T17:42:06.303171+0000","last_became_peered":"2024-09-13T17:42:06.303171+0000","last_unstale":"2024-09-13T17:42:07.326108+0000","last_undegraded":"2024-09-13T17:42:07.326108+0000","last_fullsized":"2024-09-13T17:42:07.326108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:36:02.512243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.327020+0000","last_change":"2024-09-13T17:42:07.316113+0000","last_active":"2024-09-13T17:42:07.327020+0000","last_peered":"2024-09-13T17:42:07.327020+0000","last_clean":"2024-09-13T17:42:07.327020+0000","last_became_active":"2024-09-13T17:42:07.315546+0000","last_became_peered":"2024-09-13T17:42:07.315546+0000","last_unstale":"2024-09-13T17:42:07.327020+0000","last_undegraded":"2024-09-13T17:42:07.327020+0000","last_fullsized":"2024-09-13T17:42:07.327020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:30:30.503115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.335532+0000","last_change":"2024-09-13T17:42:06.305594+0000","last_active":"2024-09-13T17:42:07.335532+0000","last_peered":"2024-09-13T17:42:07.335532+0000","last_clean":"2024-09-13T17:42:07.335532+0000","last_became_active":"2024-09-13T17:42:06.305330+0000","last_became_peered":"2024-09-13T17:42:06.305330+0000","last_unstale":"2024-09-13T17:42:07.335532+0000","last_undegraded":"2024-09-13T17:42:07.335532+0000","last_fullsized":"2024-09-13T17:42:07.335532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:17:23.133953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.335793+0000","last_change":"2024-09-13T17:42:07.333147+0000","last_active":"2024-09-13T17:42:07.335793+0000","last_peered":"2024-09-13T17:42:07.335793+0000","last_clean":"2024-09-13T17:42:07.335793+0000","last_became_active":"2024-09-13T17:42:07.331637+0000","last_became_peered":"2024-09-13T17:42:07.331637+0000","last_unstale":"2024-09-13T17:42:07.335793+0000","last_undegraded":"2024-09-13T17:42:07.335793+0000","last_fullsized":"2024-09-13T17:42:07.335793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:25.632002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.328050+0000","last_change":"2024-09-13T17:42:06.312084+0000","last_active":"2024-09-13T17:42:07.328050+0000","last_peered":"2024-09-13T17:42:07.328050+0000","last_clean":"2024-09-13T17:42:07.328050+0000","last_became_active":"2024-09-13T17:42:06.311848+0000","last_became_peered":"2024-09-13T17:42:06.311848+0000","last_unstale":"2024-09-13T17:42:07.328050+0000","last_undegraded":"2024-09-13T17:42:07.328050+0000","last_fullsized":"2024-09-13T17:42:07.328050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:32.379434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.335857+0000","last_change":"2024-09-13T17:42:07.333648+0000","last_active":"2024-09-13T17:42:07.335857+0000","last_peered":"2024-09-13T17:42:07.335857+0000","last_clean":"2024-09-13T17:42:07.335857+0000","last_became_active":"2024-09-13T17:42:07.332563+0000","last_became_peered":"2024-09-13T17:42:07.332563+0000","last_unstale":"2024-09-13T17:42:07.335857+0000","last_undegraded":"2024-09-13T17:42:07.335857+0000","last_fullsized":"2024-09-13T17:42:07.335857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:06.934321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331832+0000","last_change":"2024-09-13T17:42:06.308815+0000","last_active":"2024-09-13T17:42:07.331832+0000","last_peered":"2024-09-13T17:42:07.331832+0000","last_clean":"2024-09-13T17:42:07.331832+0000","last_became_active":"2024-09-13T17:42:06.308407+0000","last_became_peered":"2024-09-13T17:42:06.308407+0000","last_unstale":"2024-09-13T17:42:07.331832+0000","last_undegraded":"2024-09-13T17:42:07.331832+0000","last_fullsized":"2024-09-13T17:42:07.331832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:19.237563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331281+0000","last_change":"2024-09-13T17:42:07.315582+0000","last_active":"2024-09-13T17:42:07.331281+0000","last_peered":"2024-09-13T17:42:07.331281+0000","last_clean":"2024-09-13T17:42:07.331281+0000","last_became_active":"2024-09-13T17:42:07.312277+0000","last_became_peered":"2024-09-13T17:42:07.312277+0000","last_unstale":"2024-09-13T17:42:07.331281+0000","last_undegraded":"2024-09-13T17:42:07.331281+0000","last_fullsized":"2024-09-13T17:42:07.331281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:59:50.392102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338321+0000","last_change":"2024-09-13T17:42:06.312228+0000","last_active":"2024-09-13T17:42:07.338321+0000","last_peered":"2024-09-13T17:42:07.338321+0000","last_clean":"2024-09-13T17:42:07.338321+0000","last_became_active":"2024-09-13T17:42:06.311930+0000","last_became_peered":"2024-09-13T17:42:06.311930+0000","last_unstale":"2024-09-13T17:42:07.338321+0000","last_undegraded":"2024-09-13T17:42:07.338321+0000","last_fullsized":"2024-09-13T17:42:07.338321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:16.993105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.330780+0000","last_change":"2024-09-13T17:42:07.330018+0000","last_active":"2024-09-13T17:42:07.330780+0000","last_peered":"2024-09-13T17:42:07.330780+0000","last_clean":"2024-09-13T17:42:07.330780+0000","last_became_active":"2024-09-13T17:42:07.329763+0000","last_became_peered":"2024-09-13T17:42:07.329763+0000","last_unstale":"2024-09-13T17:42:07.330780+0000","last_undegraded":"2024-09-13T17:42:07.330780+0000","last_fullsized":"2024-09-13T17:42:07.330780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:38:02.719445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.337820+0000","last_change":"2024-09-13T17:42:06.305609+0000","last_active":"2024-09-13T17:42:07.337820+0000","last_peered":"2024-09-13T17:42:07.337820+0000","last_clean":"2024-09-13T17:42:07.337820+0000","last_became_active":"2024-09-13T17:42:06.304878+0000","last_became_peered":"2024-09-13T17:42:06.304878+0000","last_unstale":"2024-09-13T17:42:07.337820+0000","last_undegraded":"2024-09-13T17:42:07.337820+0000","last_fullsized":"2024-09-13T17:42:07.337820+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:32.379434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705668,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297556,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074284,"statfs":{"total":96636764160,"available":96332066816,"internally_reserved":0,"allocated":200704,"data_stored":57550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]}]},{"osd":2,"up_from":9,"seq":38654705668,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297540,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074300,"statfs":{"total":96636764160,"available":96332083200,"internally_reserved":0,"allocated":184320,"data_stored":57337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]}]},{"osd":1,"up_from":9,"seq":38654705671,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298116,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073724,"statfs":{"total":96636764160,"available":96331493376,"internally_reserved":0,"allocated":774144,"data_stored":642535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]}]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298116,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073724,"statfs":{"total":96636764160,"available":96331493376,"internally_reserved":0,"allocated":774144,"data_stored":642570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:42:10.497 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-09-13T17:42:10.497 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-13T17:42:10.497 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-13T17:42:10.497 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-13T17:42:10.498 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-13T17:42:10.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.623+0000 7f879499d640 1 Processor -- start 2024-09-13T17:42:10.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.623+0000 7f879499d640 1 -- start start 2024-09-13T17:42:10.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.624+0000 7f879499d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f879012c870 0x7f8790131490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:10.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.624+0000 7f879499d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f879012cc80 0x7f87901339e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:10.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.624+0000 7f879499d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f879006a530 con 0x7f879009abb0 2024-09-13T17:42:10.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.624+0000 7f879499d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f879009af80 con 0x7f879012cc80 2024-09-13T17:42:10.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.624+0000 7f879499d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8790098be0 con 0x7f879012c870 2024-09-13T17:42:10.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.624+0000 7f878dd9b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f879012c870 0x7f8790131490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:10.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.624+0000 7f878ed9d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f879012cc80 0x7f87901339e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:10.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.624+0000 7f878e59c640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f879009abb0 0x7f879012c160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33604/0 (socket says 172.21.15.102:33604) 2024-09-13T17:42:10.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.624+0000 7f878e59c640 1 -- 172.21.15.102:0/3630030083 learned_addr learned my addr 172.21.15.102:0/3630030083 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:42:10.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.624+0000 7f878dd9b640 1 -- 172.21.15.102:0/3630030083 >> v2:172.21.15.102:3300/0 conn(0x7f879012cc80 msgr2=0x7f87901339e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:10.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.624+0000 7f878dd9b640 1 --2- 172.21.15.102:0/3630030083 >> v2:172.21.15.102:3300/0 conn(0x7f879012cc80 0x7f87901339e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.624+0000 7f878dd9b640 1 -- 172.21.15.102:0/3630030083 >> v1:172.21.15.102:6789/0 conn(0x7f879009abb0 legacy=0x7f879012c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:10.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.624+0000 7f878dd9b640 1 -- 172.21.15.102:0/3630030083 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8790098930 con 0x7f879012c870 2024-09-13T17:42:10.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.624+0000 7f878d59a640 1 -- 172.21.15.102:0/3630030083 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1594024495 0 0) 0x7f879006a530 con 0x7f879009abb0 2024-09-13T17:42:10.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.624+0000 7f878ed9d640 1 --2- 172.21.15.102:0/3630030083 >> v2:172.21.15.102:3300/0 conn(0x7f879012cc80 0x7f87901339e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:10.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.625+0000 7f878dd9b640 1 --2- 172.21.15.102:0/3630030083 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f879012c870 0x7f8790131490 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8784009880 tx=0x7f87840090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47d92dd6e73d9572 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:10.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.625+0000 7f878d59a640 1 -- 172.21.15.102:0/3630030083 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878402ac50 con 0x7f879012c870 2024-09-13T17:42:10.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.625+0000 7f878d59a640 1 -- 172.21.15.102:0/3630030083 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878402adb0 con 0x7f879012c870 2024-09-13T17:42:10.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.625+0000 7f878d59a640 1 -- 172.21.15.102:0/3630030083 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8784032670 con 0x7f879012c870 2024-09-13T17:42:10.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.625+0000 7f879499d640 1 -- 172.21.15.102:0/3630030083 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f879012c870 msgr2=0x7f8790131490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:10.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.625+0000 7f879499d640 1 --2- 172.21.15.102:0/3630030083 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f879012c870 0x7f8790131490 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8784009880 tx=0x7f87840090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.625+0000 7f879499d640 1 -- 172.21.15.102:0/3630030083 shutdown_connections 2024-09-13T17:42:10.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.625+0000 7f879499d640 1 --2- 172.21.15.102:0/3630030083 >> v2:172.21.15.102:3300/0 conn(0x7f879012cc80 0x7f87901339e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.625+0000 7f879499d640 1 --2- 172.21.15.102:0/3630030083 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f879012c870 0x7f8790131490 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.625+0000 7f879499d640 1 -- 172.21.15.102:0/3630030083 >> 172.21.15.102:0/3630030083 conn(0x7f879009e820 msgr2=0x7f879009ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:10.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.626+0000 7f879499d640 1 -- 172.21.15.102:0/3630030083 shutdown_connections 2024-09-13T17:42:10.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.626+0000 7f879499d640 1 -- 172.21.15.102:0/3630030083 wait complete. 2024-09-13T17:42:10.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.626+0000 7f879499d640 1 Processor -- start 2024-09-13T17:42:10.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.626+0000 7f879499d640 1 -- start start 2024-09-13T17:42:10.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.627+0000 7f879499d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f879009abb0 0x7f879011d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:10.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.627+0000 7f879499d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f879012c870 0x7f8790129740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:10.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.627+0000 7f879499d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f879012cc80 0x7f8790119800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:10.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.627+0000 7f879499d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879012a3a0 con 0x7f879012c870 2024-09-13T17:42:10.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.627+0000 7f879499d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87900981a0 con 0x7f879009abb0 2024-09-13T17:42:10.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.627+0000 7f879499d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f879011cbe0 con 0x7f879012cc80 2024-09-13T17:42:10.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.627+0000 7f878e59c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f879009abb0 0x7f879011d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:10.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.627+0000 7f878dd9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f879012c870 0x7f8790129740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:10.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.627+0000 7f878ed9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f879012cc80 0x7f8790119800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:10.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.627+0000 7f878e59c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f879009abb0 0x7f879011d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:55000/0 (socket says 172.21.15.102:55000) 2024-09-13T17:42:10.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.627+0000 7f878ed9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f879012cc80 0x7f8790119800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:42094/0 (socket says 172.21.15.102:42094) 2024-09-13T17:42:10.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.627+0000 7f878e59c640 1 -- 172.21.15.102:0/2000740868 learned_addr learned my addr 172.21.15.102:0/2000740868 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:10.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.627+0000 7f878dd9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f879012c870 0x7f8790129740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52640/0 (socket says 172.21.15.102:52640) 2024-09-13T17:42:10.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.627+0000 7f878dd9b640 1 -- 172.21.15.102:0/2000740868 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f879012cc80 msgr2=0x7f8790119800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:10.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.627+0000 7f878dd9b640 1 --2- 172.21.15.102:0/2000740868 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f879012cc80 0x7f8790119800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.627+0000 7f878dd9b640 1 -- 172.21.15.102:0/2000740868 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f879009abb0 msgr2=0x7f879011d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:10.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.627+0000 7f878dd9b640 1 --2- 172.21.15.102:0/2000740868 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f879009abb0 0x7f879011d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.627+0000 7f878dd9b640 1 -- 172.21.15.102:0/2000740868 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8788009a20 con 0x7f879012c870 2024-09-13T17:42:10.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.627+0000 7f878e59c640 1 --2- 172.21.15.102:0/2000740868 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f879009abb0 0x7f879011d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:10.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.627+0000 7f878ed9d640 1 --2- 172.21.15.102:0/2000740868 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f879012cc80 0x7f8790119800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:10.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.627+0000 7f878dd9b640 1 --2- 172.21.15.102:0/2000740868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f879012c870 0x7f8790129740 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8784031000 tx=0x7f8784032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:10.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.627+0000 7f877e7fc640 1 -- 172.21.15.102:0/2000740868 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878403d720 con 0x7f879012c870 2024-09-13T17:42:10.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.627+0000 7f877e7fc640 1 -- 172.21.15.102:0/2000740868 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878403d8a0 con 0x7f879012c870 2024-09-13T17:42:10.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.627+0000 7f879499d640 1 -- 172.21.15.102:0/2000740868 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8784009390 con 0x7f879012c870 2024-09-13T17:42:10.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.627+0000 7f877e7fc640 1 -- 172.21.15.102:0/2000740868 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878403dca0 con 0x7f879012c870 2024-09-13T17:42:10.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.627+0000 7f879499d640 1 -- 172.21.15.102:0/2000740868 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8790129c80 con 0x7f879012c870 2024-09-13T17:42:10.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.628+0000 7f877e7fc640 1 -- 172.21.15.102:0/2000740868 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f8784039040 con 0x7f879012c870 2024-09-13T17:42:10.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.628+0000 7f879499d640 1 -- 172.21.15.102:0/2000740868 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8790064000 con 0x7f879012c870 2024-09-13T17:42:10.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.628+0000 7f877e7fc640 1 -- 172.21.15.102:0/2000740868 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7f8784061ba0 con 0x7f879012c870 2024-09-13T17:42:10.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.633+0000 7f877e7fc640 1 -- 172.21.15.102:0/2000740868 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f8784061e60 con 0x7f879012c870 2024-09-13T17:42:10.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.825+0000 7f879499d640 1 -- 172.21.15.102:0/2000740868 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f879011d0e0 con 0x7f879012c870 2024-09-13T17:42:10.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.826+0000 7f877e7fc640 1 -- 172.21.15.102:0/2000740868 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+73559 (secure 0 0 0) 0x7f8784061e60 con 0x7f879012c870 2024-09-13T17:42:10.829 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:42:10.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.828+0000 7f879499d640 1 -- 172.21.15.102:0/2000740868 >> v1:172.21.15.102:6816/34937 conn(0x7f8770031160 legacy=0x7f8770033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:10.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.829+0000 7f879499d640 1 -- 172.21.15.102:0/2000740868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f879012c870 msgr2=0x7f8790129740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:10.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.829+0000 7f879499d640 1 --2- 172.21.15.102:0/2000740868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f879012c870 0x7f8790129740 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8784031000 tx=0x7f8784032000 comp rx=0 tx=0).stop 2024-09-13T17:42:10.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.829+0000 7f879499d640 1 -- 172.21.15.102:0/2000740868 shutdown_connections 2024-09-13T17:42:10.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.829+0000 7f879499d640 1 --2- 172.21.15.102:0/2000740868 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f879012cc80 0x7f8790119800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.829+0000 7f879499d640 1 --2- 172.21.15.102:0/2000740868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f879012c870 0x7f8790129740 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.829+0000 7f879499d640 1 --2- 172.21.15.102:0/2000740868 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f879009abb0 0x7f879011d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.829+0000 7f879499d640 1 -- 172.21.15.102:0/2000740868 >> 172.21.15.102:0/2000740868 conn(0x7f879009e820 msgr2=0x7f879009c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:10.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.829+0000 7f879499d640 1 -- 172.21.15.102:0/2000740868 shutdown_connections 2024-09-13T17:42:10.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.829+0000 7f879499d640 1 -- 172.21.15.102:0/2000740868 wait complete. 2024-09-13T17:42:10.841 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6816","nonce":34937}]},"active_addr":"172.21.15.102:6816/34937","active_change":"2024-09-13T17:41:46.407161+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.102:0","nonce":2908116049}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.102:0","nonce":3205190670}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.102:0","nonce":3429257460}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.102:0","nonce":4212379315}]}]} 2024-09-13T17:42:10.843 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-13T17:42:10.844 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T17:42:10.844 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:10.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.934+0000 7f890c8d4640 1 Processor -- start 2024-09-13T17:42:10.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.934+0000 7f890c8d4640 1 -- start start 2024-09-13T17:42:10.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.935+0000 7f890c8d4640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f890809de20 0x7f8908134680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:10.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.935+0000 7f890c8d4640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f890809e220 0x7f8908136bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:10.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.935+0000 7f890c8d4640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89080695a0 con 0x7f8908068920 2024-09-13T17:42:10.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.935+0000 7f890c8d4640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8908068cf0 con 0x7f890809de20 2024-09-13T17:42:10.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.935+0000 7f890c8d4640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f890809f090 con 0x7f890809e220 2024-09-13T17:42:10.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.935+0000 7f8905d9b640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f890809de20 0x7f8908134680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:10.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.935+0000 7f8906d9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f890809e220 0x7f8908136bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:10.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.935+0000 7f890659c640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f8908068920 0x7f890809d710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33606/0 (socket says 172.21.15.102:33606) 2024-09-13T17:42:10.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.935+0000 7f890659c640 1 -- 172.21.15.102:0/1317555500 learned_addr learned my addr 172.21.15.102:0/1317555500 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:42:10.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.935+0000 7f8905d9b640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f890809de20 0x7f8908134680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52656/0 (socket says 172.21.15.102:52656) 2024-09-13T17:42:10.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.935+0000 7f8905d9b640 1 -- 172.21.15.102:0/1317555500 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f890809e220 msgr2=0x7f8908136bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:10.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.935+0000 7f8905d9b640 1 --2- 172.21.15.102:0/1317555500 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f890809e220 0x7f8908136bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.935+0000 7f8905d9b640 1 -- 172.21.15.102:0/1317555500 >> v1:172.21.15.102:6789/0 conn(0x7f8908068920 legacy=0x7f890809d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:10.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.935+0000 7f8905d9b640 1 -- 172.21.15.102:0/1317555500 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f890809cce0 con 0x7f890809de20 2024-09-13T17:42:10.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.935+0000 7f890559a640 1 -- 172.21.15.102:0/1317555500 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3545498831 0 0) 0x7f89080695a0 con 0x7f8908068920 2024-09-13T17:42:10.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.935+0000 7f8906d9d640 1 --2- 172.21.15.102:0/1317555500 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f890809e220 0x7f8908136bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:10.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.936+0000 7f8905d9b640 1 --2- 172.21.15.102:0/1317555500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f890809de20 0x7f8908134680 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f88f0009850 tx=0x7f88f00090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a2753fcaf8701b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:10.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.936+0000 7f890559a640 1 -- 172.21.15.102:0/1317555500 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f002ac60 con 0x7f890809de20 2024-09-13T17:42:10.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.936+0000 7f890559a640 1 -- 172.21.15.102:0/1317555500 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f002adc0 con 0x7f890809de20 2024-09-13T17:42:10.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.936+0000 7f890559a640 1 -- 172.21.15.102:0/1317555500 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f0032670 con 0x7f890809de20 2024-09-13T17:42:10.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.937+0000 7f890c8d4640 1 -- 172.21.15.102:0/1317555500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f890809de20 msgr2=0x7f8908134680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:10.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.937+0000 7f890c8d4640 1 --2- 172.21.15.102:0/1317555500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f890809de20 0x7f8908134680 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f88f0009850 tx=0x7f88f00090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.937+0000 7f890c8d4640 1 -- 172.21.15.102:0/1317555500 shutdown_connections 2024-09-13T17:42:10.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.937+0000 7f890c8d4640 1 --2- 172.21.15.102:0/1317555500 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f890809e220 0x7f8908136bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.937+0000 7f890c8d4640 1 --2- 172.21.15.102:0/1317555500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f890809de20 0x7f8908134680 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.937+0000 7f890c8d4640 1 -- 172.21.15.102:0/1317555500 >> 172.21.15.102:0/1317555500 conn(0x7f890806f320 msgr2=0x7f89080a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:10.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.937+0000 7f890c8d4640 1 -- 172.21.15.102:0/1317555500 shutdown_connections 2024-09-13T17:42:10.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.937+0000 7f890c8d4640 1 -- 172.21.15.102:0/1317555500 wait complete. 2024-09-13T17:42:10.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.937+0000 7f890c8d4640 1 Processor -- start 2024-09-13T17:42:10.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.938+0000 7f890c8d4640 1 -- start start 2024-09-13T17:42:10.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.938+0000 7f890c8d4640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8908068920 0x7f8908134080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:10.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.938+0000 7f890c8d4640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f890809de20 0x7f89081345c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:10.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.938+0000 7f890c8d4640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8908134b00 0x7f8908134f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:10.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.938+0000 7f890c8d4640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890811a7b0 con 0x7f890809de20 2024-09-13T17:42:10.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.938+0000 7f890c8d4640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8908136ac0 con 0x7f8908068920 2024-09-13T17:42:10.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.938+0000 7f890c8d4640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f89081354c0 con 0x7f8908134b00 2024-09-13T17:42:10.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.938+0000 7f890659c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8908068920 0x7f8908134080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:10.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.938+0000 7f8905d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f890809de20 0x7f89081345c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:10.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.938+0000 7f8906d9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8908134b00 0x7f8908134f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:10.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.938+0000 7f8905d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f890809de20 0x7f89081345c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52664/0 (socket says 172.21.15.102:52664) 2024-09-13T17:42:10.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.938+0000 7f8906d9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8908134b00 0x7f8908134f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:42102/0 (socket says 172.21.15.102:42102) 2024-09-13T17:42:10.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.938+0000 7f890659c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8908068920 0x7f8908134080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:55014/0 (socket says 172.21.15.102:55014) 2024-09-13T17:42:10.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.938+0000 7f8905d9b640 1 -- 172.21.15.102:0/3288409244 learned_addr learned my addr 172.21.15.102:0/3288409244 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:10.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.938+0000 7f890659c640 1 -- 172.21.15.102:0/3288409244 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8908134b00 msgr2=0x7f8908134f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:10.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.938+0000 7f890659c640 1 --2- 172.21.15.102:0/3288409244 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8908134b00 0x7f8908134f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.938+0000 7f890659c640 1 -- 172.21.15.102:0/3288409244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f890809de20 msgr2=0x7f89081345c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:10.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.938+0000 7f890659c640 1 --2- 172.21.15.102:0/3288409244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f890809de20 0x7f89081345c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.938+0000 7f890659c640 1 -- 172.21.15.102:0/3288409244 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8900009390 con 0x7f8908068920 2024-09-13T17:42:10.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.939+0000 7f890659c640 1 --2- 172.21.15.102:0/3288409244 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8908068920 0x7f8908134080 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f88f8000f80 tx=0x7f88f8001a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:10.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.939+0000 7f88fe7fc640 1 -- 172.21.15.102:0/3288409244 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f800a5e0 con 0x7f8908068920 2024-09-13T17:42:10.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.939+0000 7f88fe7fc640 1 -- 172.21.15.102:0/3288409244 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f800a780 con 0x7f8908068920 2024-09-13T17:42:10.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.939+0000 7f88fe7fc640 1 -- 172.21.15.102:0/3288409244 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f800abc0 con 0x7f8908068920 2024-09-13T17:42:10.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.939+0000 7f890c8d4640 1 -- 172.21.15.102:0/3288409244 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f0009390 con 0x7f8908068920 2024-09-13T17:42:10.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.939+0000 7f890c8d4640 1 -- 172.21.15.102:0/3288409244 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89081191b0 con 0x7f8908068920 2024-09-13T17:42:10.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.939+0000 7f88fe7fc640 1 -- 172.21.15.102:0/3288409244 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f88f80070f0 con 0x7f8908068920 2024-09-13T17:42:10.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.940+0000 7f88fe7fc640 1 -- 172.21.15.102:0/3288409244 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7f88f80420c0 con 0x7f8908068920 2024-09-13T17:42:10.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.940+0000 7f890c8d4640 1 -- 172.21.15.102:0/3288409244 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8908064000 con 0x7f8908068920 2024-09-13T17:42:10.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:10.944+0000 7f88fe7fc640 1 -- 172.21.15.102:0/3288409244 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f88f80073a0 con 0x7f8908068920 2024-09-13T17:42:11.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.097+0000 7f890c8d4640 1 -- 172.21.15.102:0/3288409244 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f890809c7d0 con 0x7f8908068920 2024-09-13T17:42:11.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.098+0000 7f88fe7fc640 1 -- 172.21.15.102:0/3288409244 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+11375 (secure 0 0 0) 0x7f88f801a7d0 con 0x7f8908068920 2024-09-13T17:42:11.099 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:42:11.100 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":20,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:42:07.375548+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:04.672265+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:05.583134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:11.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.099+0000 7f890c8d4640 1 -- 172.21.15.102:0/3288409244 >> v1:172.21.15.102:6816/34937 conn(0x7f88e8031190 legacy=0x7f88e8033640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.099+0000 7f890c8d4640 1 -- 172.21.15.102:0/3288409244 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8908068920 msgr2=0x7f8908134080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.099+0000 7f890c8d4640 1 --2- 172.21.15.102:0/3288409244 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8908068920 0x7f8908134080 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f88f8000f80 tx=0x7f88f8001a00 comp rx=0 tx=0).stop 2024-09-13T17:42:11.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.100+0000 7f890c8d4640 1 -- 172.21.15.102:0/3288409244 shutdown_connections 2024-09-13T17:42:11.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.100+0000 7f890c8d4640 1 --2- 172.21.15.102:0/3288409244 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8908134b00 0x7f8908134f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.100+0000 7f890c8d4640 1 --2- 172.21.15.102:0/3288409244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f890809de20 0x7f89081345c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.100+0000 7f890c8d4640 1 --2- 172.21.15.102:0/3288409244 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8908068920 0x7f8908134080 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.100+0000 7f890c8d4640 1 -- 172.21.15.102:0/3288409244 >> 172.21.15.102:0/3288409244 conn(0x7f890806f320 msgr2=0x7f890809bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:11.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.100+0000 7f890c8d4640 1 -- 172.21.15.102:0/3288409244 shutdown_connections 2024-09-13T17:42:11.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.100+0000 7f890c8d4640 1 -- 172.21.15.102:0/3288409244 wait complete. 2024-09-13T17:42:11.112 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T17:42:11.112 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:11.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.203+0000 7f081f99d640 1 Processor -- start 2024-09-13T17:42:11.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.203+0000 7f081f99d640 1 -- start start 2024-09-13T17:42:11.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.203+0000 7f081f99d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0818097620 0x7f081809b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.203+0000 7f081f99d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f08181350d0 0x7f08181374b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.203+0000 7f081f99d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08180969a0 con 0x7f081805db50 2024-09-13T17:42:11.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.203+0000 7f081f99d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f081805df20 con 0x7f0818097620 2024-09-13T17:42:11.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.203+0000 7f081f99d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0818097200 con 0x7f08181350d0 2024-09-13T17:42:11.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.204+0000 7f081df3a640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f08181350d0 0x7f08181374b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.204+0000 7f081d739640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f081805db50 0x7f081809a910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33614/0 (socket says 172.21.15.102:33614) 2024-09-13T17:42:11.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.204+0000 7f081cf38640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0818097620 0x7f081809b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.204+0000 7f081d739640 1 -- 172.21.15.102:0/3842721697 learned_addr learned my addr 172.21.15.102:0/3842721697 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:42:11.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.204+0000 7f080ffff640 1 -- 172.21.15.102:0/3842721697 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2801893788 0 0) 0x7f08180969a0 con 0x7f081805db50 2024-09-13T17:42:11.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.204+0000 7f081df3a640 1 -- 172.21.15.102:0/3842721697 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0818097620 msgr2=0x7f081809b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:11.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.204+0000 7f081df3a640 1 --2- 172.21.15.102:0/3842721697 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0818097620 0x7f081809b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.204+0000 7f081df3a640 1 -- 172.21.15.102:0/3842721697 >> v1:172.21.15.102:6789/0 conn(0x7f081805db50 legacy=0x7f081809a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.204+0000 7f081df3a640 1 -- 172.21.15.102:0/3842721697 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0818096c40 con 0x7f08181350d0 2024-09-13T17:42:11.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.204+0000 7f081cf38640 1 --2- 172.21.15.102:0/3842721697 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0818097620 0x7f081809b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:11.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.205+0000 7f081df3a640 1 --2- 172.21.15.102:0/3842721697 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f08181350d0 0x7f08181374b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0810009ec0 tx=0x7f0810009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e300f1e09a0f8761 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.205+0000 7f080ffff640 1 -- 172.21.15.102:0/3842721697 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0810007550 con 0x7f08181350d0 2024-09-13T17:42:11.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.205+0000 7f080ffff640 1 -- 172.21.15.102:0/3842721697 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081002d050 con 0x7f08181350d0 2024-09-13T17:42:11.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.205+0000 7f080ffff640 1 -- 172.21.15.102:0/3842721697 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08100078e0 con 0x7f08181350d0 2024-09-13T17:42:11.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.205+0000 7f081f99d640 1 -- 172.21.15.102:0/3842721697 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f08181350d0 msgr2=0x7f08181374b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.205+0000 7f081f99d640 1 --2- 172.21.15.102:0/3842721697 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f08181350d0 0x7f08181374b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0810009ec0 tx=0x7f0810009780 comp rx=0 tx=0).stop 2024-09-13T17:42:11.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.205+0000 7f081f99d640 1 -- 172.21.15.102:0/3842721697 shutdown_connections 2024-09-13T17:42:11.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.205+0000 7f081f99d640 1 --2- 172.21.15.102:0/3842721697 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f08181350d0 0x7f08181374b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.205+0000 7f081f99d640 1 --2- 172.21.15.102:0/3842721697 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0818097620 0x7f081809b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.205+0000 7f081f99d640 1 -- 172.21.15.102:0/3842721697 >> 172.21.15.102:0/3842721697 conn(0x7f081809f9d0 msgr2=0x7f081809ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:11.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.205+0000 7f081f99d640 1 -- 172.21.15.102:0/3842721697 shutdown_connections 2024-09-13T17:42:11.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.205+0000 7f081f99d640 1 -- 172.21.15.102:0/3842721697 wait complete. 2024-09-13T17:42:11.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.206+0000 7f081f99d640 1 Processor -- start 2024-09-13T17:42:11.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.206+0000 7f081f99d640 1 -- start start 2024-09-13T17:42:11.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.207+0000 7f081f99d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f081805db50 0x7f08180919f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.207+0000 7f081f99d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0818097620 0x7f081808e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.207+0000 7f081f99d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f08181350d0 0x7f081808e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.207+0000 7f081f99d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081808f750 con 0x7f081805db50 2024-09-13T17:42:11.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.207+0000 7f081f99d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f081809d790 con 0x7f0818097620 2024-09-13T17:42:11.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.207+0000 7f081f99d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0818096620 con 0x7f08181350d0 2024-09-13T17:42:11.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.207+0000 7f081d739640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f081805db50 0x7f08180919f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.207+0000 7f081cf38640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0818097620 0x7f081808e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.207+0000 7f081df3a640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f08181350d0 0x7f081808e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.207+0000 7f081d739640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f081805db50 0x7f08180919f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52676/0 (socket says 172.21.15.102:52676) 2024-09-13T17:42:11.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.207+0000 7f081d739640 1 -- 172.21.15.102:0/3604105903 learned_addr learned my addr 172.21.15.102:0/3604105903 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:11.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.207+0000 7f081cf38640 1 --2- 172.21.15.102:0/3604105903 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0818097620 0x7f081808e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:55028/0 (socket says 172.21.15.102:55028) 2024-09-13T17:42:11.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.207+0000 7f081d739640 1 -- 172.21.15.102:0/3604105903 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f08181350d0 msgr2=0x7f081808e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.207+0000 7f081d739640 1 --2- 172.21.15.102:0/3604105903 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f08181350d0 0x7f081808e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.207+0000 7f081d739640 1 -- 172.21.15.102:0/3604105903 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0818097620 msgr2=0x7f081808e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.207+0000 7f081d739640 1 --2- 172.21.15.102:0/3604105903 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0818097620 0x7f081808e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.207+0000 7f081d739640 1 -- 172.21.15.102:0/3604105903 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0808009390 con 0x7f081805db50 2024-09-13T17:42:11.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.207+0000 7f081cf38640 1 --2- 172.21.15.102:0/3604105903 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0818097620 0x7f081808e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:11.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.207+0000 7f081df3a640 1 --2- 172.21.15.102:0/3604105903 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f08181350d0 0x7f081808e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:11.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.207+0000 7f081d739640 1 --2- 172.21.15.102:0/3604105903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f081805db50 0x7f08180919f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f081400c610 tx=0x7f0814007d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.207+0000 7f080d7fa640 1 -- 172.21.15.102:0/3604105903 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0814005aa0 con 0x7f081805db50 2024-09-13T17:42:11.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.207+0000 7f080d7fa640 1 -- 172.21.15.102:0/3604105903 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0814005c20 con 0x7f081805db50 2024-09-13T17:42:11.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.207+0000 7f081f99d640 1 -- 172.21.15.102:0/3604105903 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0810009a20 con 0x7f081805db50 2024-09-13T17:42:11.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.207+0000 7f080d7fa640 1 -- 172.21.15.102:0/3604105903 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08140070f0 con 0x7f081805db50 2024-09-13T17:42:11.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.208+0000 7f081f99d640 1 -- 172.21.15.102:0/3604105903 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f081808df60 con 0x7f081805db50 2024-09-13T17:42:11.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.208+0000 7f080d7fa640 1 -- 172.21.15.102:0/3604105903 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f0814007500 con 0x7f081805db50 2024-09-13T17:42:11.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.208+0000 7f080d7fa640 1 -- 172.21.15.102:0/3604105903 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7f0814041a20 con 0x7f081805db50 2024-09-13T17:42:11.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.209+0000 7f081f99d640 1 -- 172.21.15.102:0/3604105903 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0818091330 con 0x7f081805db50 2024-09-13T17:42:11.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.213+0000 7f080d7fa640 1 -- 172.21.15.102:0/3604105903 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f081401b400 con 0x7f081805db50 2024-09-13T17:42:11.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.368+0000 7f081f99d640 1 -- 172.21.15.102:0/3604105903 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0818064000 con 0x7f081805db50 2024-09-13T17:42:11.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.369+0000 7f080d7fa640 1 -- 172.21.15.102:0/3604105903 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+11375 (secure 0 0 0) 0x7f081401a360 con 0x7f081805db50 2024-09-13T17:42:11.371 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:42:11.371 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":20,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:42:07.375548+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:04.672265+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:05.583134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:11.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.370+0000 7f081f99d640 1 -- 172.21.15.102:0/3604105903 >> v1:172.21.15.102:6816/34937 conn(0x7f07fc031190 legacy=0x7f07fc033640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.370+0000 7f081f99d640 1 -- 172.21.15.102:0/3604105903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f081805db50 msgr2=0x7f08180919f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.370+0000 7f081f99d640 1 --2- 172.21.15.102:0/3604105903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f081805db50 0x7f08180919f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f081400c610 tx=0x7f0814007d00 comp rx=0 tx=0).stop 2024-09-13T17:42:11.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.371+0000 7f081f99d640 1 -- 172.21.15.102:0/3604105903 shutdown_connections 2024-09-13T17:42:11.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.371+0000 7f081f99d640 1 --2- 172.21.15.102:0/3604105903 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f08181350d0 0x7f081808e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.371+0000 7f081f99d640 1 --2- 172.21.15.102:0/3604105903 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0818097620 0x7f081808e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.371+0000 7f081f99d640 1 --2- 172.21.15.102:0/3604105903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f081805db50 0x7f08180919f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.371+0000 7f081f99d640 1 -- 172.21.15.102:0/3604105903 >> 172.21.15.102:0/3604105903 conn(0x7f081809f9d0 msgr2=0x7f08181332e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:11.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.371+0000 7f081f99d640 1 -- 172.21.15.102:0/3604105903 shutdown_connections 2024-09-13T17:42:11.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.371+0000 7f081f99d640 1 -- 172.21.15.102:0/3604105903 wait complete. 2024-09-13T17:42:11.383 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T17:42:11.384 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T17:42:11.384 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T17:42:11.385 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-13T17:42:11.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.481+0000 7f5f40817640 1 Processor -- start 2024-09-13T17:42:11.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.481+0000 7ff5486ef640 1 Processor -- start 2024-09-13T17:42:11.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.481+0000 7f5f40817640 1 -- start start 2024-09-13T17:42:11.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.481+0000 7faf98634640 1 Processor -- start 2024-09-13T17:42:11.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.481+0000 7ff5486ef640 1 -- start start 2024-09-13T17:42:11.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.481+0000 7faf98634640 1 -- start start 2024-09-13T17:42:11.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.481+0000 7f42a9ec9640 1 Processor -- start 2024-09-13T17:42:11.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7f42a9ec9640 1 -- start start 2024-09-13T17:42:11.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7f5f40817640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f3c128330 0x7f5f3c128700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7ff5486ef640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff540128330 0x7ff540128700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7faf98634640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faf9006b140 0x7faf900998c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7faf98634640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7faf901325b0 0x7faf90134990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7faf98634640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf9006ee90 con 0x7faf9009ac20 2024-09-13T17:42:11.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7faf98634640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf9006ac90 con 0x7faf901325b0 2024-09-13T17:42:11.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7faf98634640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf9009a510 con 0x7faf9006b140 2024-09-13T17:42:11.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7ff5486ef640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff54012bf50 0x7ff540133b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7ff5486ef640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff540097be0 con 0x7ff540128c40 2024-09-13T17:42:11.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7ff5486ef640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff540098e70 con 0x7ff54012bf50 2024-09-13T17:42:11.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7ff5486ef640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff540098310 con 0x7ff540128330 2024-09-13T17:42:11.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7faf96bd1640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7faf901325b0 0x7faf90134990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7faf96bd1640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7faf901325b0 0x7faf90134990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52706/0 (socket says 172.21.15.102:52706) 2024-09-13T17:42:11.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7faf96bd1640 1 -- 172.21.15.102:0/184368107 learned_addr learned my addr 172.21.15.102:0/184368107 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:11.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7faf963d0640 1 --2- 172.21.15.102:0/184368107 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faf9006b140 0x7faf900998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7ff54648b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff540128330 0x7ff540128700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.483+0000 7ff54648b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff540128330 0x7ff540128700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:55038/0 (socket says 172.21.15.102:55038) 2024-09-13T17:42:11.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.483+0000 7ff54648b640 1 -- 172.21.15.102:0/1408567429 learned_addr learned my addr 172.21.15.102:0/1408567429 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:11.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7f42a9ec9640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f42a4068920 0x7f42a409d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7f42a9ec9640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f42a409dc50 0x7f42a409e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7f42a9ec9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f42a412f310 0x7f42a412f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7f42a9ec9640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a4068cf0 con 0x7f42a409dc50 2024-09-13T17:42:11.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7f42a9ec9640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a40695a0 con 0x7f42a4068920 2024-09-13T17:42:11.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7f42a9ec9640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f42a409f090 con 0x7f42a412f310 2024-09-13T17:42:11.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7f42a3fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f42a412f310 0x7f42a412f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7f42a3fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f42a412f310 0x7f42a412f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:42154/0 (socket says 172.21.15.102:42154) 2024-09-13T17:42:11.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7f42a3fff640 1 -- 172.21.15.102:0/1360265078 learned_addr learned my addr 172.21.15.102:0/1360265078 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:11.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.483+0000 7f42a37fe640 1 --2- 172.21.15.102:0/1360265078 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f42a4068920 0x7f42a409d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.483+0000 7f42a2ffd640 1 --2- 172.21.15.102:0/1360265078 >> v2:172.21.15.102:3300/0 conn(0x7f42a409dc50 0x7f42a409e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.483+0000 7f42a3fff640 1 -- 172.21.15.102:0/1360265078 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f42a4068920 msgr2=0x7f42a409d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:11.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.483+0000 7f42a3fff640 1 --2- 172.21.15.102:0/1360265078 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f42a4068920 0x7f42a409d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.483+0000 7f42a3fff640 1 -- 172.21.15.102:0/1360265078 >> v2:172.21.15.102:3300/0 conn(0x7f42a409dc50 msgr2=0x7f42a409e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:11.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.483+0000 7f42a3fff640 1 --2- 172.21.15.102:0/1360265078 >> v2:172.21.15.102:3300/0 conn(0x7f42a409dc50 0x7f42a409e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.483+0000 7f42a3fff640 1 -- 172.21.15.102:0/1360265078 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a409cce0 con 0x7f42a412f310 2024-09-13T17:42:11.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.483+0000 7faf963d0640 1 -- 172.21.15.102:0/184368107 >> v2:172.21.15.102:3300/0 conn(0x7faf901325b0 msgr2=0x7faf90134990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:11.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.483+0000 7faf963d0640 1 --2- 172.21.15.102:0/184368107 >> v2:172.21.15.102:3300/0 conn(0x7faf901325b0 0x7faf90134990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.483+0000 7faf963d0640 1 -- 172.21.15.102:0/184368107 >> v1:172.21.15.102:6789/0 conn(0x7faf9009ac20 legacy=0x7faf90099e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:11.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.483+0000 7faf963d0640 1 -- 172.21.15.102:0/184368107 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf90099510 con 0x7faf9006b140 2024-09-13T17:42:11.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.483+0000 7ff54648b640 1 -- 172.21.15.102:0/1408567429 >> v2:172.21.15.102:3300/0 conn(0x7ff54012bf50 msgr2=0x7ff540133b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:11.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.483+0000 7ff54648b640 1 --2- 172.21.15.102:0/1408567429 >> v2:172.21.15.102:3300/0 conn(0x7ff54012bf50 0x7ff540133b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.483+0000 7ff54648b640 1 -- 172.21.15.102:0/1408567429 >> v1:172.21.15.102:6789/0 conn(0x7ff540128c40 legacy=0x7ff54012b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:11.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.483+0000 7ff54648b640 1 -- 172.21.15.102:0/1408567429 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5400985d0 con 0x7ff540128330 2024-09-13T17:42:11.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7f5f40817640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5f3c128c40 0x7f5f3c12b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7f5f40817640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f3c0991b0 0x7f5f3c12bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7f5f40817640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f3c097be0 con 0x7f5f3c128c40 2024-09-13T17:42:11.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7f5f40817640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c098e70 con 0x7f5f3c0991b0 2024-09-13T17:42:11.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7f5f3ad9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f3c0991b0 0x7f5f3c12bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7f5f3ad9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f3c0991b0 0x7f5f3c12bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:55040/0 (socket says 172.21.15.102:55040) 2024-09-13T17:42:11.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7f5f3ad9d640 1 -- 172.21.15.102:0/3363121489 learned_addr learned my addr 172.21.15.102:0/3363121489 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:11.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7f5f3a59c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f3c128330 0x7f5f3c128700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.482+0000 7f5f40817640 1 -- 172.21.15.102:0/3363121489 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c098310 con 0x7f5f3c128330 2024-09-13T17:42:11.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.483+0000 7f5f39d9b640 1 --2- 172.21.15.102:0/3363121489 >> v2:172.21.15.102:3300/0 conn(0x7f5f3c128c40 0x7f5f3c12b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.483+0000 7f5f3ad9d640 1 -- 172.21.15.102:0/3363121489 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f3c128330 msgr2=0x7f5f3c128700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:11.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.483+0000 7f5f3ad9d640 1 --2- 172.21.15.102:0/3363121489 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f3c128330 0x7f5f3c128700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.483+0000 7f5f3ad9d640 1 -- 172.21.15.102:0/3363121489 >> v2:172.21.15.102:3300/0 conn(0x7f5f3c128c40 msgr2=0x7f5f3c12b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:11.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.483+0000 7f5f3ad9d640 1 --2- 172.21.15.102:0/3363121489 >> v2:172.21.15.102:3300/0 conn(0x7f5f3c128c40 0x7f5f3c12b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.483+0000 7f5f3ad9d640 1 -- 172.21.15.102:0/3363121489 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3c0985d0 con 0x7f5f3c0991b0 2024-09-13T17:42:11.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.483+0000 7f5f3a59c640 1 --2- 172.21.15.102:0/3363121489 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f3c128330 0x7f5f3c128700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:42:11.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.484+0000 7f42a3fff640 1 --2- 172.21.15.102:0/1360265078 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f42a412f310 0x7f42a412f790 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f429c014ac0 tx=0x7f429c0143c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d26105def61037bf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.484+0000 7ff54648b640 1 --2- 172.21.15.102:0/1408567429 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff540128330 0x7ff540128700 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009910 tx=0x7ff53c0091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53b83eeb7b760f0c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.484+0000 7f42a27fc640 1 -- 172.21.15.102:0/1360265078 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c0110d0 con 0x7f42a412f310 2024-09-13T17:42:11.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.484+0000 7f42a27fc640 1 -- 172.21.15.102:0/1360265078 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429c011230 con 0x7f42a412f310 2024-09-13T17:42:11.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.484+0000 7faf963d0640 1 --2- 172.21.15.102:0/184368107 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faf9006b140 0x7faf900998c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7faf8c009850 tx=0x7faf8c0090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe5c60674c0cab77 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.484+0000 7f42a27fc640 1 -- 172.21.15.102:0/1360265078 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c011610 con 0x7f42a412f310 2024-09-13T17:42:11.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.484+0000 7ff545489640 1 -- 172.21.15.102:0/1408567429 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c02adb0 con 0x7ff540128330 2024-09-13T17:42:11.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.484+0000 7ff545489640 1 -- 172.21.15.102:0/1408567429 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53c032440 con 0x7ff540128330 2024-09-13T17:42:11.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.484+0000 7ff545489640 1 -- 172.21.15.102:0/1408567429 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c032820 con 0x7ff540128330 2024-09-13T17:42:11.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.484+0000 7faf953ce640 1 -- 172.21.15.102:0/184368107 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c02abd0 con 0x7faf9006b140 2024-09-13T17:42:11.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.484+0000 7faf953ce640 1 -- 172.21.15.102:0/184368107 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf8c02ad30 con 0x7faf9006b140 2024-09-13T17:42:11.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.484+0000 7faf953ce640 1 -- 172.21.15.102:0/184368107 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c032670 con 0x7faf9006b140 2024-09-13T17:42:11.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.484+0000 7f5f3ad9d640 1 --2- 172.21.15.102:0/3363121489 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f3c0991b0 0x7f5f3c12bd80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5f34009f80 tx=0x7f5f34009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac018ca7036edae6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.485+0000 7f42a9ec9640 1 -- 172.21.15.102:0/1360265078 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f42a412f310 msgr2=0x7f42a412f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.485+0000 7f42a9ec9640 1 --2- 172.21.15.102:0/1360265078 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f42a412f310 0x7f42a412f790 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f429c014ac0 tx=0x7f429c0143c0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.485+0000 7f5f3959a640 1 -- 172.21.15.102:0/3363121489 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f34007400 con 0x7f5f3c0991b0 2024-09-13T17:42:11.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.485+0000 7f42a9ec9640 1 -- 172.21.15.102:0/1360265078 shutdown_connections 2024-09-13T17:42:11.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.485+0000 7f42a9ec9640 1 --2- 172.21.15.102:0/1360265078 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f42a412f310 0x7f42a412f790 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.485+0000 7f42a9ec9640 1 --2- 172.21.15.102:0/1360265078 >> v2:172.21.15.102:3300/0 conn(0x7f42a409dc50 0x7f42a409e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.485+0000 7f42a9ec9640 1 --2- 172.21.15.102:0/1360265078 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f42a4068920 0x7f42a409d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.485+0000 7f42a9ec9640 1 -- 172.21.15.102:0/1360265078 >> 172.21.15.102:0/1360265078 conn(0x7f42a406f320 msgr2=0x7f42a40a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:11.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.485+0000 7f5f3959a640 1 -- 172.21.15.102:0/3363121489 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f34007560 con 0x7f5f3c0991b0 2024-09-13T17:42:11.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.485+0000 7ff5486ef640 1 -- 172.21.15.102:0/1408567429 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff540128330 msgr2=0x7ff540128700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.485+0000 7ff5486ef640 1 --2- 172.21.15.102:0/1408567429 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff540128330 0x7ff540128700 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009910 tx=0x7ff53c0091d0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.485+0000 7f5f3959a640 1 -- 172.21.15.102:0/3363121489 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f34007900 con 0x7f5f3c0991b0 2024-09-13T17:42:11.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.485+0000 7f42a9ec9640 1 -- 172.21.15.102:0/1360265078 shutdown_connections 2024-09-13T17:42:11.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.485+0000 7f42a9ec9640 1 -- 172.21.15.102:0/1360265078 wait complete. 2024-09-13T17:42:11.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.485+0000 7ff5486ef640 1 -- 172.21.15.102:0/1408567429 shutdown_connections 2024-09-13T17:42:11.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.485+0000 7faf98634640 1 -- 172.21.15.102:0/184368107 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faf9006b140 msgr2=0x7faf900998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.485+0000 7ff5486ef640 1 --2- 172.21.15.102:0/1408567429 >> v2:172.21.15.102:3300/0 conn(0x7ff54012bf50 0x7ff540133b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.485+0000 7faf98634640 1 --2- 172.21.15.102:0/184368107 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faf9006b140 0x7faf900998c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7faf8c009850 tx=0x7faf8c0090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.485+0000 7ff5486ef640 1 --2- 172.21.15.102:0/1408567429 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff540128330 0x7ff540128700 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.485+0000 7ff5486ef640 1 -- 172.21.15.102:0/1408567429 >> 172.21.15.102:0/1408567429 conn(0x7ff54009e820 msgr2=0x7ff54009ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:11.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.485+0000 7ff5486ef640 1 -- 172.21.15.102:0/1408567429 shutdown_connections 2024-09-13T17:42:11.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.485+0000 7faf98634640 1 -- 172.21.15.102:0/184368107 shutdown_connections 2024-09-13T17:42:11.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.485+0000 7ff5486ef640 1 -- 172.21.15.102:0/1408567429 wait complete. 2024-09-13T17:42:11.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.485+0000 7faf98634640 1 --2- 172.21.15.102:0/184368107 >> v2:172.21.15.102:3300/0 conn(0x7faf901325b0 0x7faf90134990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.485+0000 7faf98634640 1 --2- 172.21.15.102:0/184368107 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faf9006b140 0x7faf900998c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.485+0000 7faf98634640 1 -- 172.21.15.102:0/184368107 >> 172.21.15.102:0/184368107 conn(0x7faf9009e820 msgr2=0x7faf9009ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:11.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.486+0000 7f5f40817640 1 -- 172.21.15.102:0/3363121489 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f3c0991b0 msgr2=0x7f5f3c12bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.486+0000 7f5f40817640 1 --2- 172.21.15.102:0/3363121489 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f3c0991b0 0x7f5f3c12bd80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5f34009f80 tx=0x7f5f34009840 comp rx=0 tx=0).stop 2024-09-13T17:42:11.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.486+0000 7f5f40817640 1 -- 172.21.15.102:0/3363121489 shutdown_connections 2024-09-13T17:42:11.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.486+0000 7f5f40817640 1 --2- 172.21.15.102:0/3363121489 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f3c0991b0 0x7f5f3c12bd80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.486+0000 7f5f40817640 1 --2- 172.21.15.102:0/3363121489 >> v2:172.21.15.102:3300/0 conn(0x7f5f3c128c40 0x7f5f3c12b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.486+0000 7f5f40817640 1 --2- 172.21.15.102:0/3363121489 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f3c128330 0x7f5f3c128700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.486+0000 7f5f40817640 1 -- 172.21.15.102:0/3363121489 >> 172.21.15.102:0/3363121489 conn(0x7f5f3c09e820 msgr2=0x7f5f3c09ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:11.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.486+0000 7f5f40817640 1 -- 172.21.15.102:0/3363121489 shutdown_connections 2024-09-13T17:42:11.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.486+0000 7f5f40817640 1 -- 172.21.15.102:0/3363121489 wait complete. 2024-09-13T17:42:11.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.486+0000 7faf98634640 1 -- 172.21.15.102:0/184368107 shutdown_connections 2024-09-13T17:42:11.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.486+0000 7faf98634640 1 -- 172.21.15.102:0/184368107 wait complete. 2024-09-13T17:42:11.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.486+0000 7f42a9ec9640 1 Processor -- start 2024-09-13T17:42:11.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.487+0000 7f42a9ec9640 1 -- start start 2024-09-13T17:42:11.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.487+0000 7f5f40817640 1 Processor -- start 2024-09-13T17:42:11.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.487+0000 7f5f40817640 1 -- start start 2024-09-13T17:42:11.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.487+0000 7ff5486ef640 1 Processor -- start 2024-09-13T17:42:11.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.487+0000 7ff5486ef640 1 -- start start 2024-09-13T17:42:11.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.487+0000 7faf98634640 1 Processor -- start 2024-09-13T17:42:11.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.487+0000 7faf98634640 1 -- start start 2024-09-13T17:42:11.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.487+0000 7f42a9ec9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f42a4068920 0x7f42a412ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.487+0000 7f42a9ec9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f42a409dc50 0x7f42a4119c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.487+0000 7f42a9ec9640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f42a412f310 0x7f42a411a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.487+0000 7f42a9ec9640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a411bcd0 con 0x7f42a409dc50 2024-09-13T17:42:11.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.487+0000 7f42a9ec9640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a412f150 con 0x7f42a412f310 2024-09-13T17:42:11.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.487+0000 7f42a9ec9640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f42a411b910 con 0x7f42a4068920 2024-09-13T17:42:11.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.487+0000 7f42a37fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f42a4068920 0x7f42a412ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.487+0000 7f42a37fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f42a4068920 0x7f42a412ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:42166/0 (socket says 172.21.15.102:42166) 2024-09-13T17:42:11.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.487+0000 7f42a37fe640 1 -- 172.21.15.102:0/3816586718 learned_addr learned my addr 172.21.15.102:0/3816586718 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:11.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.487+0000 7f5f40817640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f3c0991b0 0x7f5f3c114e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.487+0000 7f5f40817640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f3c128330 0x7f5f3c1153c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.487+0000 7f5f40817640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f3c128c40 0x7f5f3c115900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.487+0000 7f5f40817640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c126060 con 0x7f5f3c128330 2024-09-13T17:42:11.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.487+0000 7f5f40817640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c09d9c0 con 0x7f5f3c0991b0 2024-09-13T17:42:11.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.487+0000 7f5f40817640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c117e80 con 0x7f5f3c128c40 2024-09-13T17:42:11.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.487+0000 7f5f3a59c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f3c0991b0 0x7f5f3c114e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.487+0000 7f5f39d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f3c128330 0x7f5f3c1153c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.487+0000 7f5f3ad9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f3c128c40 0x7f5f3c115900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7f5f39d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f3c128330 0x7f5f3c1153c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52728/0 (socket says 172.21.15.102:52728) 2024-09-13T17:42:11.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7f5f3a59c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f3c0991b0 0x7f5f3c114e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:55054/0 (socket says 172.21.15.102:55054) 2024-09-13T17:42:11.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7f5f39d9b640 1 -- 172.21.15.102:0/2169106513 learned_addr learned my addr 172.21.15.102:0/2169106513 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:11.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7f42a2ffd640 1 --2- 172.21.15.102:0/3816586718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f42a409dc50 0x7f42a4119c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7ff5486ef640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff540128330 0x7ff54012f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7faf98634640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faf9006b140 0x7faf9011f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7ff5486ef640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff540128c40 0x7ff540137110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7faf98634640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf9009ac20 0x7faf9011fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7faf98634640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faf901325b0 0x7faf90120230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7faf963d0640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faf9006b140 0x7faf9011f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7faf98634640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf90098b30 con 0x7faf9009ac20 2024-09-13T17:42:11.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7faf98634640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf90121770 con 0x7faf9006b140 2024-09-13T17:42:11.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7faf963d0640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faf9006b140 0x7faf9011f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:55060/0 (socket says 172.21.15.102:55060) 2024-09-13T17:42:11.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7faf963d0640 1 -- 172.21.15.102:0/3381341632 learned_addr learned my addr 172.21.15.102:0/3381341632 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:11.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7faf95bcf640 1 --2- 172.21.15.102:0/3381341632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf9009ac20 0x7faf9011fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7faf98634640 1 -- 172.21.15.102:0/3381341632 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf90121360 con 0x7faf901325b0 2024-09-13T17:42:11.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7f5f3ad9d640 1 -- 172.21.15.102:0/2169106513 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f3c0991b0 msgr2=0x7f5f3c114e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7f5f3ad9d640 1 --2- 172.21.15.102:0/2169106513 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f3c0991b0 0x7f5f3c114e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7f5f3ad9d640 1 -- 172.21.15.102:0/2169106513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f3c128330 msgr2=0x7f5f3c1153c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7f5f3ad9d640 1 --2- 172.21.15.102:0/2169106513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f3c128330 0x7f5f3c1153c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7f5f3ad9d640 1 -- 172.21.15.102:0/2169106513 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f34009ae0 con 0x7f5f3c128c40 2024-09-13T17:42:11.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7f5f3a59c640 1 --2- 172.21.15.102:0/2169106513 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f3c0991b0 0x7f5f3c114e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:11.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7f5f3ad9d640 1 --2- 172.21.15.102:0/2169106513 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f3c128c40 0x7f5f3c115900 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5f34041000 tx=0x7f5f34045000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7f42a3fff640 1 --2- 172.21.15.102:0/3816586718 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f42a412f310 0x7f42a411a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7f42a37fe640 1 -- 172.21.15.102:0/3816586718 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f42a412f310 msgr2=0x7f42a411a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7f42a37fe640 1 --2- 172.21.15.102:0/3816586718 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f42a412f310 0x7f42a411a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7f42a37fe640 1 -- 172.21.15.102:0/3816586718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f42a409dc50 msgr2=0x7f42a4119c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7f42a37fe640 1 --2- 172.21.15.102:0/3816586718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f42a409dc50 0x7f42a4119c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7f5f2a7fc640 1 -- 172.21.15.102:0/2169106513 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f340032b0 con 0x7f5f3c128c40 2024-09-13T17:42:11.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7f42a37fe640 1 -- 172.21.15.102:0/3816586718 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f429c014600 con 0x7f42a4068920 2024-09-13T17:42:11.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7faf96bd1640 1 --2- 172.21.15.102:0/3381341632 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faf901325b0 0x7faf90120230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7faf963d0640 1 -- 172.21.15.102:0/3381341632 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faf901325b0 msgr2=0x7faf90120230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7faf963d0640 1 --2- 172.21.15.102:0/3381341632 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faf901325b0 0x7faf90120230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7faf963d0640 1 -- 172.21.15.102:0/3381341632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf9009ac20 msgr2=0x7faf9011fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7faf963d0640 1 --2- 172.21.15.102:0/3381341632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf9009ac20 0x7faf9011fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7faf963d0640 1 -- 172.21.15.102:0/3381341632 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf8c009390 con 0x7faf9006b140 2024-09-13T17:42:11.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7faf95bcf640 1 --2- 172.21.15.102:0/3381341632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf9009ac20 0x7faf9011fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:42:11.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7faf96bd1640 1 --2- 172.21.15.102:0/3381341632 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faf901325b0 0x7faf90120230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:42:11.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7f5f2a7fc640 1 -- 172.21.15.102:0/2169106513 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f34043e30 con 0x7f5f3c128c40 2024-09-13T17:42:11.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.489+0000 7f5f2a7fc640 1 -- 172.21.15.102:0/2169106513 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3404e640 con 0x7f5f3c128c40 2024-09-13T17:42:11.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.489+0000 7ff54648b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff540128330 0x7ff54012f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.488+0000 7f42a2ffd640 1 --2- 172.21.15.102:0/3816586718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f42a409dc50 0x7f42a4119c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:11.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.489+0000 7f42a37fe640 1 --2- 172.21.15.102:0/3816586718 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f42a4068920 0x7f42a412ec10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f429400d890 tx=0x7f4294008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.489+0000 7f5f40817640 1 -- 172.21.15.102:0/2169106513 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f3c125ca0 con 0x7f5f3c128c40 2024-09-13T17:42:11.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.489+0000 7f5f40817640 1 -- 172.21.15.102:0/2169106513 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f3c1259d0 con 0x7f5f3c128c40 2024-09-13T17:42:11.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.489+0000 7faf963d0640 1 --2- 172.21.15.102:0/3381341632 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faf9006b140 0x7faf9011f7b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faf8c009850 tx=0x7faf8c032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.489+0000 7faf867fc640 1 -- 172.21.15.102:0/3381341632 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c02aea0 con 0x7faf9006b140 2024-09-13T17:42:11.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.489+0000 7faf867fc640 1 -- 172.21.15.102:0/3381341632 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf8c031a60 con 0x7faf9006b140 2024-09-13T17:42:11.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.489+0000 7faf98634640 1 -- 172.21.15.102:0/3381341632 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf90122e80 con 0x7faf9006b140 2024-09-13T17:42:11.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.489+0000 7f428b7fe640 1 -- 172.21.15.102:0/3816586718 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429400de20 con 0x7f42a4068920 2024-09-13T17:42:11.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.489+0000 7f42a9ec9640 1 -- 172.21.15.102:0/3816586718 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a411d100 con 0x7f42a4068920 2024-09-13T17:42:11.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.489+0000 7f428b7fe640 1 -- 172.21.15.102:0/3816586718 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42940098b0 con 0x7f42a4068920 2024-09-13T17:42:11.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.489+0000 7f42a9ec9640 1 -- 172.21.15.102:0/3816586718 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a411bf10 con 0x7f42a4068920 2024-09-13T17:42:11.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.489+0000 7ff5486ef640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff54012bf50 0x7ff54006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.489+0000 7ff5486ef640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54012d1c0 con 0x7ff540128330 2024-09-13T17:42:11.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.489+0000 7ff545c8a640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff540128c40 0x7ff540137110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.489+0000 7ff545c8a640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff540128c40 0x7ff540137110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:55066/0 (socket says 172.21.15.102:55066) 2024-09-13T17:42:11.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.489+0000 7ff545c8a640 1 -- 172.21.15.102:0/3246488354 learned_addr learned my addr 172.21.15.102:0/3246488354 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:11.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.489+0000 7ff546c8c640 1 --2- 172.21.15.102:0/3246488354 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff54012bf50 0x7ff54006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.490+0000 7f5f40817640 1 -- 172.21.15.102:0/2169106513 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5f3c064000 con 0x7f5f3c128c40 2024-09-13T17:42:11.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.490+0000 7f428b7fe640 1 -- 172.21.15.102:0/3816586718 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429400de20 con 0x7f42a4068920 2024-09-13T17:42:11.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.490+0000 7f5f2a7fc640 1 -- 172.21.15.102:0/2169106513 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f5f3404ea30 con 0x7f5f3c128c40 2024-09-13T17:42:11.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.489+0000 7faf867fc640 1 -- 172.21.15.102:0/3381341632 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c031e20 con 0x7faf9006b140 2024-09-13T17:42:11.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.489+0000 7faf98634640 1 -- 172.21.15.102:0/3381341632 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf901228a0 con 0x7faf9006b140 2024-09-13T17:42:11.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.490+0000 7ff5486ef640 1 -- 172.21.15.102:0/3246488354 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff54012db90 con 0x7ff540128c40 2024-09-13T17:42:11.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.490+0000 7ff5486ef640 1 -- 172.21.15.102:0/3246488354 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff54012e8b0 con 0x7ff54012bf50 2024-09-13T17:42:11.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.490+0000 7faf98634640 1 -- 172.21.15.102:0/3381341632 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faf90064000 con 0x7faf9006b140 2024-09-13T17:42:11.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.490+0000 7f428b7fe640 1 -- 172.21.15.102:0/3816586718 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f4294009c70 con 0x7f42a4068920 2024-09-13T17:42:11.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.490+0000 7f5f2a7fc640 1 -- 172.21.15.102:0/2169106513 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7f5f34074880 con 0x7f5f3c128c40 2024-09-13T17:42:11.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.490+0000 7f42897fa640 1 -- 172.21.15.102:0/3816586718 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f42a4064000 con 0x7f42a4068920 2024-09-13T17:42:11.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.490+0000 7f5f2a7fc640 1 -- 172.21.15.102:0/2169106513 --> v1:172.21.15.102:6800/35327 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5f1c03dad0 con 0x7f5f1c039e10 2024-09-13T17:42:11.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.490+0000 7f5f2a7fc640 1 -- 172.21.15.102:0/2169106513 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f34079220 con 0x7f5f3c128c40 2024-09-13T17:42:11.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.491+0000 7f428b7fe640 1 -- 172.21.15.102:0/3816586718 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7f429401b6b0 con 0x7f42a4068920 2024-09-13T17:42:11.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.490+0000 7ff54648b640 1 -- 172.21.15.102:0/3246488354 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff54012bf50 msgr2=0x7ff54006d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.491+0000 7ff54648b640 1 --2- 172.21.15.102:0/3246488354 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff54012bf50 0x7ff54006d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.491+0000 7ff54648b640 1 -- 172.21.15.102:0/3246488354 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff540128c40 msgr2=0x7ff540137110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.491+0000 7ff54648b640 1 --2- 172.21.15.102:0/3246488354 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff540128c40 0x7ff540137110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.491+0000 7ff54648b640 1 -- 172.21.15.102:0/3246488354 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff53c009450 con 0x7ff540128330 2024-09-13T17:42:11.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.491+0000 7f428b7fe640 1 -- 172.21.15.102:0/3816586718 --> v1:172.21.15.102:6812/35352 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f429003e1a0 con 0x7f429003a4a0 2024-09-13T17:42:11.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.491+0000 7ff545c8a640 1 --2- 172.21.15.102:0/3246488354 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff540128c40 0x7ff540137110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:11.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.491+0000 7f428b7fe640 1 -- 172.21.15.102:0/3816586718 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f429400b3d0 con 0x7f42a4068920 2024-09-13T17:42:11.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.491+0000 7ff54648b640 1 --2- 172.21.15.102:0/3246488354 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff540128330 0x7ff54012f090 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff540098450 tx=0x7ff53c032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.491+0000 7ff546c8c640 1 --2- 172.21.15.102:0/3246488354 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff54012bf50 0x7ff54006d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:11.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.491+0000 7ff5367fc640 1 -- 172.21.15.102:0/3246488354 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c03d440 con 0x7ff540128330 2024-09-13T17:42:11.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.491+0000 7faf867fc640 1 -- 172.21.15.102:0/3381341632 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7faf8c039040 con 0x7faf9006b140 2024-09-13T17:42:11.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.491+0000 7ff5486ef640 1 -- 172.21.15.102:0/3246488354 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54012d400 con 0x7ff540128330 2024-09-13T17:42:11.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.491+0000 7ff5367fc640 1 -- 172.21.15.102:0/3246488354 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53c03d5a0 con 0x7ff540128330 2024-09-13T17:42:11.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.491+0000 7ff5367fc640 1 -- 172.21.15.102:0/3246488354 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c03d920 con 0x7ff540128330 2024-09-13T17:42:11.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.491+0000 7ff5486ef640 1 -- 172.21.15.102:0/3246488354 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff540137650 con 0x7ff540128330 2024-09-13T17:42:11.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.491+0000 7faf867fc640 1 -- 172.21.15.102:0/3381341632 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7faf8c062630 con 0x7faf9006b140 2024-09-13T17:42:11.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.491+0000 7faf867fc640 1 -- 172.21.15.102:0/3381341632 --> v1:172.21.15.102:6804/35328 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faf7403dc20 con 0x7faf74039f60 2024-09-13T17:42:11.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.491+0000 7faf867fc640 1 -- 172.21.15.102:0/3381341632 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf8c062960 con 0x7faf9006b140 2024-09-13T17:42:11.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.492+0000 7f5f2a7fc640 1 -- 172.21.15.102:0/2169106513 <== osd.0 v1:172.21.15.102:6800/35327 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f5f1c03dad0 con 0x7f5f1c039e10 2024-09-13T17:42:11.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.492+0000 7ff5367fc640 1 -- 172.21.15.102:0/3246488354 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7ff53c039040 con 0x7ff540128330 2024-09-13T17:42:11.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.492+0000 7ff5486ef640 1 -- 172.21.15.102:0/3246488354 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff540137840 con 0x7ff540128330 2024-09-13T17:42:11.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.493+0000 7ff5367fc640 1 -- 172.21.15.102:0/3246488354 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7ff53c0626d0 con 0x7ff540128330 2024-09-13T17:42:11.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.493+0000 7f428b7fe640 1 -- 172.21.15.102:0/3816586718 <== osd.3 v1:172.21.15.102:6812/35352 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f429003e1a0 con 0x7f429003a4a0 2024-09-13T17:42:11.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.493+0000 7ff5367fc640 1 -- 172.21.15.102:0/3246488354 --> v1:172.21.15.102:6808/35351 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff52403d9d0 con 0x7ff524039d10 2024-09-13T17:42:11.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.493+0000 7ff5367fc640 1 -- 172.21.15.102:0/3246488354 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff53c064c30 con 0x7ff540128330 2024-09-13T17:42:11.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.493+0000 7faf867fc640 1 -- 172.21.15.102:0/3381341632 <== osd.1 v1:172.21.15.102:6804/35328 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7faf7403dc20 con 0x7faf74039f60 2024-09-13T17:42:11.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.494+0000 7ff5367fc640 1 -- 172.21.15.102:0/3246488354 <== osd.2 v1:172.21.15.102:6808/35351 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7ff52403d9d0 con 0x7ff524039d10 2024-09-13T17:42:11.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.505+0000 7faf98634640 1 -- 172.21.15.102:0/3381341632 --> v1:172.21.15.102:6804/35328 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faf90064000 con 0x7faf74039f60 2024-09-13T17:42:11.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.506+0000 7faf867fc640 1 -- 172.21.15.102:0/3381341632 <== osd.1 v1:172.21.15.102:6804/35328 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 2724425506) 0x7faf90064000 con 0x7faf74039f60 2024-09-13T17:42:11.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.506+0000 7faf98634640 1 -- 172.21.15.102:0/3381341632 >> v1:172.21.15.102:6804/35328 conn(0x7faf74039f60 legacy=0x7faf7403c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.506+0000 7ff5486ef640 1 -- 172.21.15.102:0/3246488354 --> v1:172.21.15.102:6808/35351 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff540129a00 con 0x7ff524039d10 2024-09-13T17:42:11.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.506+0000 7faf98634640 1 -- 172.21.15.102:0/3381341632 >> v1:172.21.15.102:6816/34937 conn(0x7faf740311b0 legacy=0x7faf74033660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.506+0000 7faf98634640 1 -- 172.21.15.102:0/3381341632 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faf9006b140 msgr2=0x7faf9011f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.506+0000 7faf98634640 1 --2- 172.21.15.102:0/3381341632 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faf9006b140 0x7faf9011f7b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faf8c009850 tx=0x7faf8c032000 comp rx=0 tx=0).stop 2024-09-13T17:42:11.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.506+0000 7faf96bd1640 1 -- 172.21.15.102:0/3381341632 reap_dead start 2024-09-13T17:42:11.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.506+0000 7faf98634640 1 -- 172.21.15.102:0/3381341632 shutdown_connections 2024-09-13T17:42:11.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.506+0000 7faf98634640 1 -- 172.21.15.102:0/3381341632 >> 172.21.15.102:0/3381341632 conn(0x7faf9009e820 msgr2=0x7faf9009c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:11.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.506+0000 7faf98634640 1 -- 172.21.15.102:0/3381341632 shutdown_connections 2024-09-13T17:42:11.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.506+0000 7ff5367fc640 1 -- 172.21.15.102:0/3246488354 <== osd.2 v1:172.21.15.102:6808/35351 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 675830894) 0x7ff540129a00 con 0x7ff524039d10 2024-09-13T17:42:11.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.506+0000 7faf98634640 1 -- 172.21.15.102:0/3381341632 wait complete. 2024-09-13T17:42:11.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.507+0000 7ff5486ef640 1 -- 172.21.15.102:0/3246488354 >> v1:172.21.15.102:6808/35351 conn(0x7ff524039d10 legacy=0x7ff52403c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.507+0000 7ff5486ef640 1 -- 172.21.15.102:0/3246488354 >> v1:172.21.15.102:6816/34937 conn(0x7ff524031160 legacy=0x7ff524033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.507+0000 7ff5486ef640 1 -- 172.21.15.102:0/3246488354 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff540128330 msgr2=0x7ff54012f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.507+0000 7ff5486ef640 1 --2- 172.21.15.102:0/3246488354 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff540128330 0x7ff54012f090 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff540098450 tx=0x7ff53c032000 comp rx=0 tx=0).stop 2024-09-13T17:42:11.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.507+0000 7ff546c8c640 1 -- 172.21.15.102:0/3246488354 reap_dead start 2024-09-13T17:42:11.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.507+0000 7ff5486ef640 1 -- 172.21.15.102:0/3246488354 shutdown_connections 2024-09-13T17:42:11.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.507+0000 7ff5486ef640 1 -- 172.21.15.102:0/3246488354 >> 172.21.15.102:0/3246488354 conn(0x7ff54009e820 msgr2=0x7ff54012aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:11.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.507+0000 7ff5486ef640 1 -- 172.21.15.102:0/3246488354 shutdown_connections 2024-09-13T17:42:11.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.507+0000 7ff5486ef640 1 -- 172.21.15.102:0/3246488354 wait complete. 2024-09-13T17:42:11.548 INFO:teuthology.orchestra.run.smithi102.stdout:38654705672 2024-09-13T17:42:11.548 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T17:42:11.549 INFO:teuthology.orchestra.run.smithi102.stdout:38654705669 2024-09-13T17:42:11.550 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T17:42:11.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.520+0000 7f5f40817640 1 -- 172.21.15.102:0/2169106513 --> v1:172.21.15.102:6800/35327 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5f3c06dd20 con 0x7f5f1c039e10 2024-09-13T17:42:11.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.521+0000 7f5f2a7fc640 1 -- 172.21.15.102:0/2169106513 <== osd.0 v1:172.21.15.102:6800/35327 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 1129848789) 0x7f5f3c06dd20 con 0x7f5f1c039e10 2024-09-13T17:42:11.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.521+0000 7f42897fa640 1 -- 172.21.15.102:0/3816586718 --> v1:172.21.15.102:6812/35352 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f42a411c700 con 0x7f429003a4a0 2024-09-13T17:42:11.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.521+0000 7f5f40817640 1 -- 172.21.15.102:0/2169106513 >> v1:172.21.15.102:6800/35327 conn(0x7f5f1c039e10 legacy=0x7f5f1c03c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.521+0000 7f5f40817640 1 -- 172.21.15.102:0/2169106513 >> v1:172.21.15.102:6816/34937 conn(0x7f5f1c031160 legacy=0x7f5f1c033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.521+0000 7f5f40817640 1 -- 172.21.15.102:0/2169106513 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f3c128c40 msgr2=0x7f5f3c115900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.521+0000 7f5f40817640 1 --2- 172.21.15.102:0/2169106513 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f3c128c40 0x7f5f3c115900 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5f34041000 tx=0x7f5f34045000 comp rx=0 tx=0).stop 2024-09-13T17:42:11.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.521+0000 7f428b7fe640 1 -- 172.21.15.102:0/3816586718 <== osd.3 v1:172.21.15.102:6812/35352 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 675830894) 0x7f42a411c700 con 0x7f429003a4a0 2024-09-13T17:42:11.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.521+0000 7f5f3ad9d640 1 -- 172.21.15.102:0/2169106513 reap_dead start 2024-09-13T17:42:11.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.521+0000 7f5f40817640 1 -- 172.21.15.102:0/2169106513 shutdown_connections 2024-09-13T17:42:11.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.521+0000 7f5f40817640 1 -- 172.21.15.102:0/2169106513 >> 172.21.15.102:0/2169106513 conn(0x7f5f3c09e820 msgr2=0x7f5f3c127a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:11.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.521+0000 7f5f40817640 1 -- 172.21.15.102:0/2169106513 shutdown_connections 2024-09-13T17:42:11.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.521+0000 7f5f40817640 1 -- 172.21.15.102:0/2169106513 wait complete. 2024-09-13T17:42:11.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.522+0000 7f42897fa640 1 -- 172.21.15.102:0/3816586718 >> v1:172.21.15.102:6812/35352 conn(0x7f429003a4a0 legacy=0x7f429003c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.522+0000 7f42897fa640 1 -- 172.21.15.102:0/3816586718 >> v1:172.21.15.102:6816/34937 conn(0x7f42900311c0 legacy=0x7f4290033670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.522+0000 7f42897fa640 1 -- 172.21.15.102:0/3816586718 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f42a4068920 msgr2=0x7f42a412ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.522+0000 7f42897fa640 1 --2- 172.21.15.102:0/3816586718 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f42a4068920 0x7f42a412ec10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f429400d890 tx=0x7f4294008040 comp rx=0 tx=0).stop 2024-09-13T17:42:11.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.522+0000 7f42a3fff640 1 -- 172.21.15.102:0/3816586718 reap_dead start 2024-09-13T17:42:11.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.522+0000 7f42897fa640 1 -- 172.21.15.102:0/3816586718 shutdown_connections 2024-09-13T17:42:11.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.522+0000 7f42897fa640 1 -- 172.21.15.102:0/3816586718 >> 172.21.15.102:0/3816586718 conn(0x7f42a406f320 msgr2=0x7f42a406db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:11.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.522+0000 7f42897fa640 1 -- 172.21.15.102:0/3816586718 shutdown_connections 2024-09-13T17:42:11.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.522+0000 7f42897fa640 1 -- 172.21.15.102:0/3816586718 wait complete. 2024-09-13T17:42:11.556 INFO:teuthology.orchestra.run.smithi102.stdout:38654705670 2024-09-13T17:42:11.556 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T17:42:11.557 INFO:teuthology.orchestra.run.smithi102.stdout:38654705669 2024-09-13T17:42:11.557 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T17:42:11.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.686+0000 7f03fc657640 1 Processor -- start 2024-09-13T17:42:11.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.687+0000 7f03fc657640 1 -- start start 2024-09-13T17:42:11.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.687+0000 7f5ca4354640 1 Processor -- start 2024-09-13T17:42:11.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.687+0000 7f5ca4354640 1 -- start start 2024-09-13T17:42:11.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.687+0000 7f03fc657640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f03f4068920 0x7f03f409d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.687+0000 7f03fc657640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f03f409dc50 0x7f03f409e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.687+0000 7f03fc657640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03f40695a0 con 0x7f03f412f310 2024-09-13T17:42:11.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.687+0000 7f03fc657640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f4068cf0 con 0x7f03f409dc50 2024-09-13T17:42:11.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.687+0000 7f03fc657640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f03f409f090 con 0x7f03f4068920 2024-09-13T17:42:11.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.687+0000 7f03fa3f3640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f03f4068920 0x7f03f409d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.687+0000 7f03fabf4640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f03f412f310 0x7f03f412f7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33634/0 (socket says 172.21.15.102:33634) 2024-09-13T17:42:11.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.687+0000 7f03fabf4640 1 -- 172.21.15.102:0/1824515673 learned_addr learned my addr 172.21.15.102:0/1824515673 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:42:11.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.687+0000 7f03f9bf2640 1 --2- 172.21.15.102:0/1824515673 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f03f409dc50 0x7f03f409e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.688+0000 7f5ca4354640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5c9c06d670 0x7f5c9c06e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.688+0000 7f5ca4354640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5c9c06eb60 0x7f5c9c12d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.688+0000 7f03f93f1640 1 -- 172.21.15.102:0/1824515673 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 95805705 0 0) 0x7f03f40695a0 con 0x7f03f412f310 2024-09-13T17:42:11.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.688+0000 7f03f93f1640 1 -- 172.21.15.102:0/1824515673 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03e00034e0 con 0x7f03f412f310 2024-09-13T17:42:11.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.688+0000 7f5ca20f0640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5c9c06d670 0x7f5c9c06e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.688+0000 7f5ca20f0640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5c9c06d670 0x7f5c9c06e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:42214/0 (socket says 172.21.15.102:42214) 2024-09-13T17:42:11.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.688+0000 7f5ca18ef640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5c9c06eb60 0x7f5c9c12d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.688+0000 7f03f9bf2640 1 -- 172.21.15.102:0/1824515673 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f03f4068920 msgr2=0x7f03f409d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:11.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.688+0000 7f03f9bf2640 1 --2- 172.21.15.102:0/1824515673 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f03f4068920 0x7f03f409d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.688+0000 7f03f9bf2640 1 -- 172.21.15.102:0/1824515673 >> v1:172.21.15.102:6789/0 conn(0x7f03f412f310 legacy=0x7f03f412f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.688+0000 7f5ca28f1640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f5c9c09f7f0 0x7f5c9c12d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33646/0 (socket says 172.21.15.102:33646) 2024-09-13T17:42:11.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.688+0000 7f5ca18ef640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5c9c06eb60 0x7f5c9c12d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:55088/0 (socket says 172.21.15.102:55088) 2024-09-13T17:42:11.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.688+0000 7f5ca20f0640 1 -- 172.21.15.102:0/1582300280 learned_addr learned my addr 172.21.15.102:0/1582300280 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:11.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.688+0000 7f5ca4354640 1 -- 172.21.15.102:0/1582300280 --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c9c06c910 con 0x7f5c9c09f7f0 2024-09-13T17:42:11.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.688+0000 7f5ca4354640 1 -- 172.21.15.102:0/1582300280 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c06da40 con 0x7f5c9c06eb60 2024-09-13T17:42:11.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.688+0000 7f5ca4354640 1 -- 172.21.15.102:0/1582300280 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c06c130 con 0x7f5c9c06d670 2024-09-13T17:42:11.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.688+0000 7f03f9bf2640 1 -- 172.21.15.102:0/1824515673 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f409cce0 con 0x7f03f409dc50 2024-09-13T17:42:11.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.688+0000 7f5ca10ee640 1 -- 172.21.15.102:0/1582300280 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 112882013 0 0) 0x7f5c9c06c910 con 0x7f5c9c09f7f0 2024-09-13T17:42:11.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.688+0000 7f5ca10ee640 1 -- 172.21.15.102:0/1582300280 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c880034e0 con 0x7f5c9c09f7f0 2024-09-13T17:42:11.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.688+0000 7f5ca20f0640 1 -- 172.21.15.102:0/1582300280 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5c9c06eb60 msgr2=0x7f5c9c12d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:11.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.688+0000 7f5ca20f0640 1 --2- 172.21.15.102:0/1582300280 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5c9c06eb60 0x7f5c9c12d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.688+0000 7f5ca20f0640 1 -- 172.21.15.102:0/1582300280 >> v1:172.21.15.102:6789/0 conn(0x7f5c9c09f7f0 legacy=0x7f5c9c12d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.689+0000 7f5ca20f0640 1 -- 172.21.15.102:0/1582300280 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9c0a0310 con 0x7f5c9c06d670 2024-09-13T17:42:11.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.689+0000 7f03f9bf2640 1 --2- 172.21.15.102:0/1824515673 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f03f409dc50 0x7f03f409e020 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009880 tx=0x7f03e40090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23abf1814c497a2d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.689+0000 7f03f93f1640 1 -- 172.21.15.102:0/1824515673 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e402ace0 con 0x7f03f409dc50 2024-09-13T17:42:11.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.689+0000 7f03f93f1640 1 -- 172.21.15.102:0/1824515673 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e402ae40 con 0x7f03f409dc50 2024-09-13T17:42:11.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.689+0000 7f03f93f1640 1 -- 172.21.15.102:0/1824515673 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e4032650 con 0x7f03f409dc50 2024-09-13T17:42:11.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.689+0000 7f5ca20f0640 1 --2- 172.21.15.102:0/1582300280 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5c9c06d670 0x7f5c9c06e620 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5c98009940 tx=0x7f5c980091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2a01991ef609327 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.690+0000 7f5ca10ee640 1 -- 172.21.15.102:0/1582300280 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c98032440 con 0x7f5c9c06d670 2024-09-13T17:42:11.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.690+0000 7f5ca10ee640 1 -- 172.21.15.102:0/1582300280 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c980325a0 con 0x7f5c9c06d670 2024-09-13T17:42:11.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.690+0000 7f5ca10ee640 1 -- 172.21.15.102:0/1582300280 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c98032960 con 0x7f5c9c06d670 2024-09-13T17:42:11.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.690+0000 7f53a0aa8640 1 Processor -- start 2024-09-13T17:42:11.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.690+0000 7f53a0aa8640 1 -- start start 2024-09-13T17:42:11.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.690+0000 7f03fc657640 1 -- 172.21.15.102:0/1824515673 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f03f409dc50 msgr2=0x7f03f409e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.690+0000 7f03fc657640 1 --2- 172.21.15.102:0/1824515673 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f03f409dc50 0x7f03f409e020 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009880 tx=0x7f03e40090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.690+0000 7f03fc657640 1 -- 172.21.15.102:0/1824515673 shutdown_connections 2024-09-13T17:42:11.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.690+0000 7f03fc657640 1 --2- 172.21.15.102:0/1824515673 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f03f409dc50 0x7f03f409e020 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.690+0000 7f03fc657640 1 --2- 172.21.15.102:0/1824515673 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f03f4068920 0x7f03f409d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.690+0000 7fe3c991d640 1 Processor -- start 2024-09-13T17:42:11.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.690+0000 7f03fc657640 1 -- 172.21.15.102:0/1824515673 >> 172.21.15.102:0/1824515673 conn(0x7f03f406f320 msgr2=0x7f03f40a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:11.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.690+0000 7f03fc657640 1 -- 172.21.15.102:0/1824515673 shutdown_connections 2024-09-13T17:42:11.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.690+0000 7f03fc657640 1 -- 172.21.15.102:0/1824515673 wait complete. 2024-09-13T17:42:11.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.690+0000 7fe3c991d640 1 -- start start 2024-09-13T17:42:11.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.690+0000 7f5ca4354640 1 -- 172.21.15.102:0/1582300280 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5c9c06d670 msgr2=0x7f5c9c06e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.690+0000 7f5ca4354640 1 --2- 172.21.15.102:0/1582300280 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5c9c06d670 0x7f5c9c06e620 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5c98009940 tx=0x7f5c980091d0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.691+0000 7f53a0aa8640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f539c06d670 0x7f539c06e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.691+0000 7f53a0aa8640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f539c06eb60 0x7f539c12d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.691+0000 7f5ca4354640 1 -- 172.21.15.102:0/1582300280 shutdown_connections 2024-09-13T17:42:11.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.691+0000 7f53a0aa8640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f539c06c910 con 0x7f539c09f7f0 2024-09-13T17:42:11.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.691+0000 7f539ad9d640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f539c09f7f0 0x7f539c12d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:33660/0 (socket says 172.21.15.102:33660) 2024-09-13T17:42:11.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.691+0000 7f53a0aa8640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f539c06da40 con 0x7f539c06eb60 2024-09-13T17:42:11.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.691+0000 7f539a59c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f539c06d670 0x7f539c06e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.691+0000 7f539ad9d640 1 -- 172.21.15.102:0/3261493913 learned_addr learned my addr 172.21.15.102:0/3261493913 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:42:11.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.691+0000 7f53a0aa8640 1 -- 172.21.15.102:0/3261493913 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f539c06c130 con 0x7f539c06d670 2024-09-13T17:42:11.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.691+0000 7fe3c991d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe3c406b0e0 0x7fe3c4099a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.691+0000 7fe3c991d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe3c409a390 0x7fe3c412da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.691+0000 7fe3c991d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3c4069ee0 con 0x7fe3c4099f60 2024-09-13T17:42:11.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.691+0000 7fe3c991d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c406b4b0 con 0x7fe3c406b0e0 2024-09-13T17:42:11.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.691+0000 7fe3c991d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c406a950 con 0x7fe3c409a390 2024-09-13T17:42:11.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.691+0000 7f5ca4354640 1 --2- 172.21.15.102:0/1582300280 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5c9c06eb60 0x7f5c9c12d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.691+0000 7f5ca4354640 1 --2- 172.21.15.102:0/1582300280 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5c9c06d670 0x7f5c9c06e620 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.691+0000 7fe3c2ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe3c406b0e0 0x7fe3c4099a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.691+0000 7fe3c37fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe3c409a390 0x7fe3c412da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.691+0000 7fe3c2ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe3c406b0e0 0x7fe3c4099a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:55098/0 (socket says 172.21.15.102:55098) 2024-09-13T17:42:11.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.691+0000 7fe3c2ffd640 1 -- 172.21.15.102:0/4003661144 learned_addr learned my addr 172.21.15.102:0/4003661144 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:11.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.691+0000 7f5ca4354640 1 -- 172.21.15.102:0/1582300280 >> 172.21.15.102:0/1582300280 conn(0x7f5c9c067bb0 msgr2=0x7f5c9c0a08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:11.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.691+0000 7f03fc657640 1 Processor -- start 2024-09-13T17:42:11.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.691+0000 7f03fc657640 1 -- start start 2024-09-13T17:42:11.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.691+0000 7f5399d9b640 1 --2- 172.21.15.102:0/3261493913 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f539c06eb60 0x7f539c12d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.691+0000 7f539a59c640 1 -- 172.21.15.102:0/3261493913 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f539c06eb60 msgr2=0x7f539c12d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:11.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.691+0000 7f539a59c640 1 --2- 172.21.15.102:0/3261493913 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f539c06eb60 0x7f539c12d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.691+0000 7f539a59c640 1 -- 172.21.15.102:0/3261493913 >> v1:172.21.15.102:6789/0 conn(0x7f539c09f7f0 legacy=0x7f539c12d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.691+0000 7f539a59c640 1 -- 172.21.15.102:0/3261493913 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f539c0a0310 con 0x7f539c06d670 2024-09-13T17:42:11.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.691+0000 7f539959a640 1 -- 172.21.15.102:0/3261493913 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 489353895 0 0) 0x7f539c06c910 con 0x7f539c09f7f0 2024-09-13T17:42:11.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.692+0000 7fe3c37fe640 1 -- 172.21.15.102:0/4003661144 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe3c406b0e0 msgr2=0x7fe3c4099a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:11.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.692+0000 7fe3c37fe640 1 --2- 172.21.15.102:0/4003661144 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe3c406b0e0 0x7fe3c4099a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.692+0000 7fe3c37fe640 1 -- 172.21.15.102:0/4003661144 >> v1:172.21.15.102:6789/0 conn(0x7fe3c4099f60 legacy=0x7fe3c412d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.692+0000 7fe3c37fe640 1 -- 172.21.15.102:0/4003661144 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c409a830 con 0x7fe3c409a390 2024-09-13T17:42:11.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.692+0000 7fe3c2ffd640 1 --2- 172.21.15.102:0/4003661144 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe3c406b0e0 0x7fe3c4099a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:11.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.692+0000 7f5ca4354640 1 -- 172.21.15.102:0/1582300280 shutdown_connections 2024-09-13T17:42:11.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.692+0000 7f5ca4354640 1 -- 172.21.15.102:0/1582300280 wait complete. 2024-09-13T17:42:11.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.692+0000 7f03fc657640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f03f4068920 0x7f03f4125e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.692+0000 7f03fc657640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03f409dc50 0x7f03f4122140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.692+0000 7fe3c37fe640 1 --2- 172.21.15.102:0/4003661144 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe3c409a390 0x7fe3c412da00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc009ec0 tx=0x7fe3bc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5db7e0f32f45079e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.692+0000 7f03fc657640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f03f412f310 0x7f03f4122680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.692+0000 7f03fc657640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f409fcb0 con 0x7f03f409dc50 2024-09-13T17:42:11.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.692+0000 7f03fc657640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f409e620 con 0x7f03f412f310 2024-09-13T17:42:11.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.693+0000 7fe3c1ffb640 1 -- 172.21.15.102:0/4003661144 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc007550 con 0x7fe3c409a390 2024-09-13T17:42:11.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.692+0000 7f03fc657640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f03f4123f20 con 0x7f03f4068920 2024-09-13T17:42:11.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.692+0000 7f03fa3f3640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f03f4068920 0x7f03f4125e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.692+0000 7f03fa3f3640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f03f4068920 0x7f03f4125e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:42242/0 (socket says 172.21.15.102:42242) 2024-09-13T17:42:11.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.692+0000 7f03fa3f3640 1 -- 172.21.15.102:0/1412648319 learned_addr learned my addr 172.21.15.102:0/1412648319 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:11.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.693+0000 7fe3c1ffb640 1 -- 172.21.15.102:0/4003661144 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3bc02d050 con 0x7fe3c409a390 2024-09-13T17:42:11.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.693+0000 7f03fabf4640 1 --2- 172.21.15.102:0/1412648319 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f03f412f310 0x7f03f4122680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.693+0000 7f03f9bf2640 1 --2- 172.21.15.102:0/1412648319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03f409dc50 0x7f03f4122140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.693+0000 7fe3c1ffb640 1 -- 172.21.15.102:0/4003661144 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc0078e0 con 0x7fe3c409a390 2024-09-13T17:42:11.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.693+0000 7f539a59c640 1 --2- 172.21.15.102:0/3261493913 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f539c06d670 0x7f539c06e620 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f538c009880 tx=0x7f538c0090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ccbadbd0cca5c02 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.693+0000 7f539959a640 1 -- 172.21.15.102:0/3261493913 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c036660 con 0x7f539c06d670 2024-09-13T17:42:11.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.693+0000 7f539959a640 1 -- 172.21.15.102:0/3261493913 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f538c0367c0 con 0x7f539c06d670 2024-09-13T17:42:11.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.693+0000 7f539959a640 1 -- 172.21.15.102:0/3261493913 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c036ba0 con 0x7f539c06d670 2024-09-13T17:42:11.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.693+0000 7f03fa3f3640 1 -- 172.21.15.102:0/1412648319 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f03f412f310 msgr2=0x7f03f4122680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.693+0000 7f03fa3f3640 1 --2- 172.21.15.102:0/1412648319 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f03f412f310 0x7f03f4122680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.693+0000 7f03fa3f3640 1 -- 172.21.15.102:0/1412648319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03f409dc50 msgr2=0x7f03f4122140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.693+0000 7f03fa3f3640 1 --2- 172.21.15.102:0/1412648319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03f409dc50 0x7f03f4122140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.693+0000 7f03fa3f3640 1 -- 172.21.15.102:0/1412648319 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e4009390 con 0x7f03f4068920 2024-09-13T17:42:11.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.693+0000 7f03fabf4640 1 --2- 172.21.15.102:0/1412648319 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f03f412f310 0x7f03f4122680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:42:11.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.693+0000 7f03f9bf2640 1 --2- 172.21.15.102:0/1412648319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03f409dc50 0x7f03f4122140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:42:11.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.693+0000 7f5ca4354640 1 Processor -- start 2024-09-13T17:42:11.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.693+0000 7f5ca4354640 1 -- start start 2024-09-13T17:42:11.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.693+0000 7f03fa3f3640 1 --2- 172.21.15.102:0/1412648319 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f03f4068920 0x7f03f4125e60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f03f000ea50 tx=0x7f03f00090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.693+0000 7f03ea7fc640 1 -- 172.21.15.102:0/1412648319 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f0009600 con 0x7f03f4068920 2024-09-13T17:42:11.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.693+0000 7f03ea7fc640 1 -- 172.21.15.102:0/1412648319 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f000ca10 con 0x7f03f4068920 2024-09-13T17:42:11.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.693+0000 7f03fc657640 1 -- 172.21.15.102:0/1412648319 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03f4124ed0 con 0x7f03f4068920 2024-09-13T17:42:11.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.694+0000 7f03ea7fc640 1 -- 172.21.15.102:0/1412648319 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f000cd90 con 0x7f03f4068920 2024-09-13T17:42:11.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.694+0000 7f03fc657640 1 -- 172.21.15.102:0/1412648319 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03f4123c70 con 0x7f03f4068920 2024-09-13T17:42:11.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.694+0000 7fe3c991d640 1 -- 172.21.15.102:0/4003661144 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe3c409a390 msgr2=0x7fe3c412da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.694+0000 7fe3c991d640 1 --2- 172.21.15.102:0/4003661144 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe3c409a390 0x7fe3c412da00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc009ec0 tx=0x7fe3bc009780 comp rx=0 tx=0).stop 2024-09-13T17:42:11.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.694+0000 7f53a0aa8640 1 -- 172.21.15.102:0/3261493913 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f539c06d670 msgr2=0x7f539c06e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.694+0000 7f53a0aa8640 1 --2- 172.21.15.102:0/3261493913 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f539c06d670 0x7f539c06e620 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f538c009880 tx=0x7f538c0090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.694+0000 7f5ca4354640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5c9c06d670 0x7f5c9c126c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.694+0000 7f5ca4354640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5c9c06eb60 0x7f5c9c122eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.694+0000 7f5ca4354640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c9c09f7f0 0x7f5c9c1233f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.694+0000 7fe3c991d640 1 -- 172.21.15.102:0/4003661144 shutdown_connections 2024-09-13T17:42:11.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.694+0000 7fe3c991d640 1 --2- 172.21.15.102:0/4003661144 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe3c409a390 0x7fe3c412da00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.694+0000 7fe3c991d640 1 --2- 172.21.15.102:0/4003661144 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe3c406b0e0 0x7fe3c4099a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.694+0000 7fe3c991d640 1 -- 172.21.15.102:0/4003661144 >> 172.21.15.102:0/4003661144 conn(0x7fe3c409ea30 msgr2=0x7fe3c409ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:11.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.694+0000 7f5ca20f0640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5c9c06d670 0x7f5c9c126c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.694+0000 7f5ca4354640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c124db0 con 0x7f5c9c09f7f0 2024-09-13T17:42:11.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.694+0000 7f5ca4354640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c06c290 con 0x7f5c9c06d670 2024-09-13T17:42:11.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.694+0000 7f5ca18ef640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5c9c06eb60 0x7f5c9c122eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.694+0000 7f5ca20f0640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5c9c06d670 0x7f5c9c126c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:55110/0 (socket says 172.21.15.102:55110) 2024-09-13T17:42:11.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.694+0000 7f5ca20f0640 1 -- 172.21.15.102:0/2792194278 learned_addr learned my addr 172.21.15.102:0/2792194278 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:11.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.694+0000 7f5ca4354640 1 -- 172.21.15.102:0/2792194278 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c125d80 con 0x7f5c9c06eb60 2024-09-13T17:42:11.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.694+0000 7f5ca28f1640 1 --2- 172.21.15.102:0/2792194278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c9c09f7f0 0x7f5c9c1233f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.694+0000 7f53a0aa8640 1 -- 172.21.15.102:0/3261493913 shutdown_connections 2024-09-13T17:42:11.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.694+0000 7f53a0aa8640 1 --2- 172.21.15.102:0/3261493913 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f539c06eb60 0x7f539c12d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.694+0000 7f5ca18ef640 1 -- 172.21.15.102:0/2792194278 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5c9c06d670 msgr2=0x7f5c9c126c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.694+0000 7f5ca18ef640 1 --2- 172.21.15.102:0/2792194278 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5c9c06d670 0x7f5c9c126c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.694+0000 7f53a0aa8640 1 --2- 172.21.15.102:0/3261493913 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f539c06d670 0x7f539c06e620 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.694+0000 7f53a0aa8640 1 -- 172.21.15.102:0/3261493913 >> 172.21.15.102:0/3261493913 conn(0x7f539c067bb0 msgr2=0x7f539c0a08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:11.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.694+0000 7f5ca18ef640 1 -- 172.21.15.102:0/2792194278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c9c09f7f0 msgr2=0x7f5c9c1233f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.694+0000 7f5ca18ef640 1 --2- 172.21.15.102:0/2792194278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c9c09f7f0 0x7f5c9c1233f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.694+0000 7f5ca18ef640 1 -- 172.21.15.102:0/2792194278 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c98009450 con 0x7f5c9c06eb60 2024-09-13T17:42:11.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.694+0000 7f5ca20f0640 1 --2- 172.21.15.102:0/2792194278 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5c9c06d670 0x7f5c9c126c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:11.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.695+0000 7f53a0aa8640 1 -- 172.21.15.102:0/3261493913 shutdown_connections 2024-09-13T17:42:11.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.695+0000 7f5ca18ef640 1 --2- 172.21.15.102:0/2792194278 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5c9c06eb60 0x7f5c9c122eb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00d8f0 tx=0x7f5c8c008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.695+0000 7f53a0aa8640 1 -- 172.21.15.102:0/3261493913 wait complete. 2024-09-13T17:42:11.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.695+0000 7f5c927fc640 1 -- 172.21.15.102:0/2792194278 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c00de80 con 0x7f5c9c06eb60 2024-09-13T17:42:11.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.695+0000 7f5ca4354640 1 -- 172.21.15.102:0/2792194278 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c9c1264a0 con 0x7f5c9c06eb60 2024-09-13T17:42:11.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.695+0000 7f5c927fc640 1 -- 172.21.15.102:0/2792194278 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c8c00c940 con 0x7f5c9c06eb60 2024-09-13T17:42:11.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.695+0000 7f5c927fc640 1 -- 172.21.15.102:0/2792194278 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c00ccc0 con 0x7f5c9c06eb60 2024-09-13T17:42:11.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.695+0000 7fe3c991d640 1 -- 172.21.15.102:0/4003661144 shutdown_connections 2024-09-13T17:42:11.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.695+0000 7f5ca4354640 1 -- 172.21.15.102:0/2792194278 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c9c123930 con 0x7f5c9c06eb60 2024-09-13T17:42:11.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.695+0000 7f03ea7fc640 1 -- 172.21.15.102:0/1412648319 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f03f0013680 con 0x7f03f4068920 2024-09-13T17:42:11.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.695+0000 7f03ea7fc640 1 -- 172.21.15.102:0/1412648319 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7f03f00427e0 con 0x7f03f4068920 2024-09-13T17:42:11.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.695+0000 7fe3c991d640 1 -- 172.21.15.102:0/4003661144 wait complete. 2024-09-13T17:42:11.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.695+0000 7f03fc657640 1 -- 172.21.15.102:0/1412648319 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03f4125320 con 0x7f03f4068920 2024-09-13T17:42:11.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.696+0000 7f53a0aa8640 1 Processor -- start 2024-09-13T17:42:11.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.696+0000 7f5c927fc640 1 -- 172.21.15.102:0/2792194278 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f5c8c015620 con 0x7f5c9c06eb60 2024-09-13T17:42:11.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.696+0000 7f5c927fc640 1 -- 172.21.15.102:0/2792194278 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7f5c8c041bb0 con 0x7f5c9c06eb60 2024-09-13T17:42:11.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.696+0000 7f5ca4354640 1 -- 172.21.15.102:0/2792194278 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c9c1261c0 con 0x7f5c9c06eb60 2024-09-13T17:42:11.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.696+0000 7f53a0aa8640 1 -- start start 2024-09-13T17:42:11.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.696+0000 7f53a0aa8640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f539c06d670 0x7f539c13a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.696+0000 7f53a0aa8640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f539c06eb60 0x7f539c13aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.696+0000 7f53a0aa8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f539c09f7f0 0x7f539c13b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.696+0000 7f53a0aa8640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539c09b700 con 0x7f539c09f7f0 2024-09-13T17:42:11.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.696+0000 7f53a0aa8640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f539c11ce60 con 0x7f539c06eb60 2024-09-13T17:42:11.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.696+0000 7f53a0aa8640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f539c11b9b0 con 0x7f539c06d670 2024-09-13T17:42:11.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.697+0000 7f539a59c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f539c06d670 0x7f539c13a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.697+0000 7f539a59c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f539c06d670 0x7f539c13a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:42258/0 (socket says 172.21.15.102:42258) 2024-09-13T17:42:11.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.697+0000 7f539a59c640 1 -- 172.21.15.102:0/1460863149 learned_addr learned my addr 172.21.15.102:0/1460863149 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:11.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.697+0000 7fe3c991d640 1 Processor -- start 2024-09-13T17:42:11.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.697+0000 7f539ad9d640 1 --2- 172.21.15.102:0/1460863149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f539c09f7f0 0x7f539c13b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.697+0000 7f5399d9b640 1 --2- 172.21.15.102:0/1460863149 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f539c06eb60 0x7f539c13aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.697+0000 7f539ad9d640 1 -- 172.21.15.102:0/1460863149 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f539c06d670 msgr2=0x7f539c13a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.697+0000 7f539ad9d640 1 --2- 172.21.15.102:0/1460863149 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f539c06d670 0x7f539c13a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.697+0000 7f539ad9d640 1 -- 172.21.15.102:0/1460863149 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f539c06eb60 msgr2=0x7f539c13aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.697+0000 7f539ad9d640 1 --2- 172.21.15.102:0/1460863149 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f539c06eb60 0x7f539c13aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.697+0000 7f539ad9d640 1 -- 172.21.15.102:0/1460863149 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538c009390 con 0x7f539c09f7f0 2024-09-13T17:42:11.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.697+0000 7fe3c991d640 1 -- start start 2024-09-13T17:42:11.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.697+0000 7f539ad9d640 1 --2- 172.21.15.102:0/1460863149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f539c09f7f0 0x7f539c13b1e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f539400cbe0 tx=0x7f53940019a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.697+0000 7f538a7fc640 1 -- 172.21.15.102:0/1460863149 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539400ce00 con 0x7f539c09f7f0 2024-09-13T17:42:11.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.697+0000 7f538a7fc640 1 -- 172.21.15.102:0/1460863149 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f539400a850 con 0x7f539c09f7f0 2024-09-13T17:42:11.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.697+0000 7f53a0aa8640 1 -- 172.21.15.102:0/1460863149 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f539c11d890 con 0x7f539c09f7f0 2024-09-13T17:42:11.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.697+0000 7f538a7fc640 1 -- 172.21.15.102:0/1460863149 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539400ac90 con 0x7f539c09f7f0 2024-09-13T17:42:11.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.697+0000 7f539a59c640 1 --2- 172.21.15.102:0/1460863149 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f539c06d670 0x7f539c13a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:11.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.697+0000 7f53a0aa8640 1 -- 172.21.15.102:0/1460863149 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f539c119d40 con 0x7f539c09f7f0 2024-09-13T17:42:11.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.698+0000 7fe3c991d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe3c406b0e0 0x7fe3c413a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.698+0000 7fe3c991d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3c4099f60 0x7fe3c413ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.698+0000 7fe3c991d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe3c409a390 0x7fe3c4119520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.698+0000 7fe3c991d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c411b900 con 0x7fe3c4099f60 2024-09-13T17:42:11.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.698+0000 7fe3c991d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c409d940 con 0x7fe3c406b0e0 2024-09-13T17:42:11.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.699+0000 7f538a7fc640 1 -- 172.21.15.102:0/1460863149 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f5394007300 con 0x7f539c09f7f0 2024-09-13T17:42:11.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.698+0000 7fe3c991d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c406ea80 con 0x7fe3c409a390 2024-09-13T17:42:11.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.698+0000 7fe3c2ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe3c406b0e0 0x7fe3c413a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.698+0000 7fe3c2ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe3c406b0e0 0x7fe3c413a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:55116/0 (socket says 172.21.15.102:55116) 2024-09-13T17:42:11.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.698+0000 7fe3c2ffd640 1 -- 172.21.15.102:0/1394648979 learned_addr learned my addr 172.21.15.102:0/1394648979 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:11.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.698+0000 7fe3c37fe640 1 --2- 172.21.15.102:0/1394648979 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe3c409a390 0x7fe3c4119520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.698+0000 7fe3c27fc640 1 --2- 172.21.15.102:0/1394648979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3c4099f60 0x7fe3c413ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.698+0000 7fe3c37fe640 1 -- 172.21.15.102:0/1394648979 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe3c406b0e0 msgr2=0x7fe3c413a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.698+0000 7fe3c37fe640 1 --2- 172.21.15.102:0/1394648979 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe3c406b0e0 0x7fe3c413a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.698+0000 7fe3c37fe640 1 -- 172.21.15.102:0/1394648979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3c4099f60 msgr2=0x7fe3c413ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.698+0000 7fe3c37fe640 1 --2- 172.21.15.102:0/1394648979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3c4099f60 0x7fe3c413ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.698+0000 7fe3c37fe640 1 -- 172.21.15.102:0/1394648979 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b4009450 con 0x7fe3c409a390 2024-09-13T17:42:11.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.698+0000 7fe3c27fc640 1 --2- 172.21.15.102:0/1394648979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3c4099f60 0x7fe3c413ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:42:11.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.698+0000 7fe3c2ffd640 1 --2- 172.21.15.102:0/1394648979 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe3c406b0e0 0x7fe3c413a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:11.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.699+0000 7fe3c37fe640 1 --2- 172.21.15.102:0/1394648979 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe3c409a390 0x7fe3c4119520 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc031000 tx=0x7fe3bc037000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.699+0000 7f53a0aa8640 1 -- 172.21.15.102:0/1460863149 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f539c06c300 con 0x7f539c09f7f0 2024-09-13T17:42:11.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.699+0000 7f538a7fc640 1 -- 172.21.15.102:0/1460863149 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7f53940421b0 con 0x7f539c09f7f0 2024-09-13T17:42:11.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.702+0000 7f03ea7fc640 1 -- 172.21.15.102:0/1412648319 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f03f0021070 con 0x7f03f4068920 2024-09-13T17:42:11.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.703+0000 7f538a7fc640 1 -- 172.21.15.102:0/1460863149 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f539c06c300 con 0x7f539c09f7f0 2024-09-13T17:42:11.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.710+0000 7f5c927fc640 1 -- 172.21.15.102:0/2792194278 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f5c8c01a020 con 0x7f5c9c06eb60 2024-09-13T17:42:11.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.710+0000 7fe3aaffd640 1 -- 172.21.15.102:0/1394648979 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc03d730 con 0x7fe3c409a390 2024-09-13T17:42:11.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.710+0000 7fe3c991d640 1 -- 172.21.15.102:0/1394648979 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3bc009a20 con 0x7fe3c409a390 2024-09-13T17:42:11.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.710+0000 7fe3aaffd640 1 -- 172.21.15.102:0/1394648979 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3bc03d8d0 con 0x7fe3c409a390 2024-09-13T17:42:11.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.710+0000 7fe3aaffd640 1 -- 172.21.15.102:0/1394648979 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc03dc90 con 0x7fe3c409a390 2024-09-13T17:42:11.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.710+0000 7fe3c991d640 1 -- 172.21.15.102:0/1394648979 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c411c9a0 con 0x7fe3c409a390 2024-09-13T17:42:11.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.711+0000 7fe3aaffd640 1 -- 172.21.15.102:0/1394648979 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fe3bc043260 con 0x7fe3c409a390 2024-09-13T17:42:11.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.711+0000 7fe3aaffd640 1 -- 172.21.15.102:0/1394648979 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7fe3bc062eb0 con 0x7fe3c409a390 2024-09-13T17:42:11.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.711+0000 7fe3c991d640 1 -- 172.21.15.102:0/1394648979 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a4005210 con 0x7fe3c409a390 2024-09-13T17:42:11.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.720+0000 7fe3aaffd640 1 -- 172.21.15.102:0/1394648979 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fe3bc042080 con 0x7fe3c409a390 2024-09-13T17:42:11.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.871+0000 7f03fc657640 1 -- 172.21.15.102:0/1412648319 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f03f41255e0 con 0x7f03f4068920 2024-09-13T17:42:11.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.871+0000 7f5ca4354640 1 -- 172.21.15.102:0/2792194278 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5c9c064600 con 0x7f5c9c06eb60 2024-09-13T17:42:11.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.871+0000 7f03ea7fc640 1 -- 172.21.15.102:0/1412648319 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f03f002a090 con 0x7f03f4068920 2024-09-13T17:42:11.874 INFO:teuthology.orchestra.run.smithi102.stdout:38654705671 2024-09-13T17:42:11.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.872+0000 7f5c927fc640 1 -- 172.21.15.102:0/2792194278 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5c8c027070 con 0x7f5c9c06eb60 2024-09-13T17:42:11.875 INFO:teuthology.orchestra.run.smithi102.stdout:38654705669 2024-09-13T17:42:11.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.873+0000 7f03fc657640 1 -- 172.21.15.102:0/1412648319 >> v1:172.21.15.102:6816/34937 conn(0x7f03d8031160 legacy=0x7f03d8033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.873+0000 7f03fc657640 1 -- 172.21.15.102:0/1412648319 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f03f4068920 msgr2=0x7f03f4125e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.873+0000 7f03fc657640 1 --2- 172.21.15.102:0/1412648319 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f03f4068920 0x7f03f4125e60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f03f000ea50 tx=0x7f03f00090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.873+0000 7f03fc657640 1 -- 172.21.15.102:0/1412648319 shutdown_connections 2024-09-13T17:42:11.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.873+0000 7f03fc657640 1 --2- 172.21.15.102:0/1412648319 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f03f412f310 0x7f03f4122680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.873+0000 7f03fc657640 1 --2- 172.21.15.102:0/1412648319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03f409dc50 0x7f03f4122140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.873+0000 7f03fc657640 1 --2- 172.21.15.102:0/1412648319 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f03f4068920 0x7f03f4125e60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.873+0000 7f03fc657640 1 -- 172.21.15.102:0/1412648319 >> 172.21.15.102:0/1412648319 conn(0x7f03f406f320 msgr2=0x7f03f40a1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:11.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.874+0000 7f03fc657640 1 -- 172.21.15.102:0/1412648319 shutdown_connections 2024-09-13T17:42:11.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.874+0000 7f03fc657640 1 -- 172.21.15.102:0/1412648319 wait complete. 2024-09-13T17:42:11.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.875+0000 7f5ca4354640 1 -- 172.21.15.102:0/2792194278 >> v1:172.21.15.102:6816/34937 conn(0x7f5c80031160 legacy=0x7f5c80033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.875+0000 7f5ca4354640 1 -- 172.21.15.102:0/2792194278 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5c9c06eb60 msgr2=0x7f5c9c122eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.875+0000 7f5ca4354640 1 --2- 172.21.15.102:0/2792194278 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5c9c06eb60 0x7f5c9c122eb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00d8f0 tx=0x7f5c8c008040 comp rx=0 tx=0).stop 2024-09-13T17:42:11.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.875+0000 7f5ca4354640 1 -- 172.21.15.102:0/2792194278 shutdown_connections 2024-09-13T17:42:11.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.875+0000 7f5ca4354640 1 --2- 172.21.15.102:0/2792194278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c9c09f7f0 0x7f5c9c1233f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.875+0000 7f5ca4354640 1 --2- 172.21.15.102:0/2792194278 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5c9c06eb60 0x7f5c9c122eb0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.875+0000 7f5ca4354640 1 --2- 172.21.15.102:0/2792194278 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5c9c06d670 0x7f5c9c126c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.875+0000 7f5ca4354640 1 -- 172.21.15.102:0/2792194278 >> 172.21.15.102:0/2792194278 conn(0x7f5c9c067bb0 msgr2=0x7f5c9c09d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:11.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.875+0000 7f5ca4354640 1 -- 172.21.15.102:0/2792194278 shutdown_connections 2024-09-13T17:42:11.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.875+0000 7f5ca4354640 1 -- 172.21.15.102:0/2792194278 wait complete. 2024-09-13T17:42:11.887 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705672 got 38654705671 for osd.1 2024-09-13T17:42:11.893 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705669 for osd.0 2024-09-13T17:42:11.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.943+0000 7f53a0aa8640 1 -- 172.21.15.102:0/1460863149 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f539c06c300 con 0x7f539c09f7f0 2024-09-13T17:42:11.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.943+0000 7f538a7fc640 1 -- 172.21.15.102:0/1460863149 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f539c06c300 con 0x7f539c09f7f0 2024-09-13T17:42:11.945 INFO:teuthology.orchestra.run.smithi102.stdout:38654705668 2024-09-13T17:42:11.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.945+0000 7f53a0aa8640 1 -- 172.21.15.102:0/1460863149 >> v1:172.21.15.102:6816/34937 conn(0x7f537c031160 legacy=0x7f537c033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.945+0000 7f53a0aa8640 1 -- 172.21.15.102:0/1460863149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f539c09f7f0 msgr2=0x7f539c13b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.945+0000 7f53a0aa8640 1 --2- 172.21.15.102:0/1460863149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f539c09f7f0 0x7f539c13b1e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f539400cbe0 tx=0x7f53940019a0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.945+0000 7f53a0aa8640 1 -- 172.21.15.102:0/1460863149 shutdown_connections 2024-09-13T17:42:11.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.945+0000 7f53a0aa8640 1 --2- 172.21.15.102:0/1460863149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f539c09f7f0 0x7f539c13b1e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.945+0000 7f53a0aa8640 1 --2- 172.21.15.102:0/1460863149 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f539c06eb60 0x7f539c13aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.945+0000 7f53a0aa8640 1 --2- 172.21.15.102:0/1460863149 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f539c06d670 0x7f539c13a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.945+0000 7f53a0aa8640 1 -- 172.21.15.102:0/1460863149 >> 172.21.15.102:0/1460863149 conn(0x7f539c067bb0 msgr2=0x7f539c12d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:11.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.945+0000 7f53a0aa8640 1 -- 172.21.15.102:0/1460863149 shutdown_connections 2024-09-13T17:42:11.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.945+0000 7f53a0aa8640 1 -- 172.21.15.102:0/1460863149 wait complete. 2024-09-13T17:42:11.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.951+0000 7fe3c991d640 1 -- 172.21.15.102:0/1394648979 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe3a4005790 con 0x7fe3c409a390 2024-09-13T17:42:11.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.951+0000 7fe3aaffd640 1 -- 172.21.15.102:0/1394648979 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe3bc0305d0 con 0x7fe3c409a390 2024-09-13T17:42:11.953 INFO:teuthology.orchestra.run.smithi102.stdout:38654705668 2024-09-13T17:42:11.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.953+0000 7fe3c991d640 1 -- 172.21.15.102:0/1394648979 >> v1:172.21.15.102:6816/34937 conn(0x7fe3b0031160 legacy=0x7fe3b0033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.953+0000 7fe3c991d640 1 -- 172.21.15.102:0/1394648979 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe3c409a390 msgr2=0x7fe3c4119520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.953+0000 7fe3c991d640 1 --2- 172.21.15.102:0/1394648979 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe3c409a390 0x7fe3c4119520 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc031000 tx=0x7fe3bc037000 comp rx=0 tx=0).stop 2024-09-13T17:42:11.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.953+0000 7fe3c991d640 1 -- 172.21.15.102:0/1394648979 shutdown_connections 2024-09-13T17:42:11.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.953+0000 7fe3c991d640 1 --2- 172.21.15.102:0/1394648979 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe3c409a390 0x7fe3c4119520 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.953+0000 7fe3c991d640 1 --2- 172.21.15.102:0/1394648979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3c4099f60 0x7fe3c413ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.953+0000 7fe3c991d640 1 --2- 172.21.15.102:0/1394648979 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe3c406b0e0 0x7fe3c413a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.953+0000 7fe3c991d640 1 -- 172.21.15.102:0/1394648979 >> 172.21.15.102:0/1394648979 conn(0x7fe3c409ea30 msgr2=0x7fe3c409c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:11.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.953+0000 7fe3c991d640 1 -- 172.21.15.102:0/1394648979 shutdown_connections 2024-09-13T17:42:11.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:11.953+0000 7fe3c991d640 1 -- 172.21.15.102:0/1394648979 wait complete. 2024-09-13T17:42:11.958 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705668 for osd.2 2024-09-13T17:42:11.965 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705668 for osd.3 2024-09-13T17:42:12.278 INFO:tasks.ceph.mds.a.smithi102.stderr:2024-09-13T17:42:12.276+0000 7fc94bbf1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:42:12.888 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T17:42:12.893 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T17:42:12.958 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T17:42:12.966 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T17:42:12.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.986+0000 7f68c175d640 1 Processor -- start 2024-09-13T17:42:12.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.986+0000 7f68c175d640 1 -- start start 2024-09-13T17:42:12.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.986+0000 7f68c175d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f68bc097670 0x7f68bc09a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:12.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.987+0000 7f68c175d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f68bc09b180 0x7f68bc1376f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:12.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.987+0000 7f68c175d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68bc095970 con 0x7f68bc09ad50 2024-09-13T17:42:12.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.987+0000 7f68c175d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68bc0971c0 con 0x7f68bc097670 2024-09-13T17:42:12.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.987+0000 7f68c175d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68bc097a40 con 0x7f68bc09b180 2024-09-13T17:42:12.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.987+0000 7f68bb7fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f68bc09b180 0x7f68bc1376f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:12.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.987+0000 7f68bb7fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f68bc09b180 0x7f68bc1376f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35276/0 (socket says 172.21.15.102:35276) 2024-09-13T17:42:12.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.987+0000 7f68ba7fc640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f68bc09ad50 0x7f68bc133fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:60776/0 (socket says 172.21.15.102:60776) 2024-09-13T17:42:12.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.987+0000 7f68bb7fe640 1 -- 172.21.15.102:0/2593454900 learned_addr learned my addr 172.21.15.102:0/2593454900 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:12.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.987+0000 7f68baffd640 1 --2- 172.21.15.102:0/2593454900 >> v2:172.21.15.102:3300/0 conn(0x7f68bc097670 0x7f68bc09a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:12.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.987+0000 7f68bb7fe640 1 -- 172.21.15.102:0/2593454900 >> v2:172.21.15.102:3300/0 conn(0x7f68bc097670 msgr2=0x7f68bc09a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:12.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.987+0000 7f68bb7fe640 1 --2- 172.21.15.102:0/2593454900 >> v2:172.21.15.102:3300/0 conn(0x7f68bc097670 0x7f68bc09a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:12.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.987+0000 7f68bb7fe640 1 -- 172.21.15.102:0/2593454900 >> v1:172.21.15.102:6789/0 conn(0x7f68bc09ad50 legacy=0x7f68bc133fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:12.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.987+0000 7f68bb7fe640 1 -- 172.21.15.102:0/2593454900 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68bc096960 con 0x7f68bc09b180 2024-09-13T17:42:12.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.988+0000 7f68bb7fe640 1 --2- 172.21.15.102:0/2593454900 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f68bc09b180 0x7f68bc1376f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f68b4009f20 tx=0x7f68b4009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7c2c8bc0b09a826 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:12.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.988+0000 7f68b9ffb640 1 -- 172.21.15.102:0/2593454900 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b4031400 con 0x7f68bc09b180 2024-09-13T17:42:12.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.988+0000 7f68b9ffb640 1 -- 172.21.15.102:0/2593454900 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b402d050 con 0x7f68bc09b180 2024-09-13T17:42:12.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.988+0000 7f68b9ffb640 1 -- 172.21.15.102:0/2593454900 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b40317d0 con 0x7f68bc09b180 2024-09-13T17:42:12.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.989+0000 7f68c175d640 1 -- 172.21.15.102:0/2593454900 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f68bc09b180 msgr2=0x7f68bc1376f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:12.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.989+0000 7f68c175d640 1 --2- 172.21.15.102:0/2593454900 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f68bc09b180 0x7f68bc1376f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f68b4009f20 tx=0x7f68b4009840 comp rx=0 tx=0).stop 2024-09-13T17:42:12.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.989+0000 7f68c175d640 1 -- 172.21.15.102:0/2593454900 shutdown_connections 2024-09-13T17:42:12.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.989+0000 7f68c175d640 1 --2- 172.21.15.102:0/2593454900 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f68bc09b180 0x7f68bc1376f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:12.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.989+0000 7f68c175d640 1 --2- 172.21.15.102:0/2593454900 >> v2:172.21.15.102:3300/0 conn(0x7f68bc097670 0x7f68bc09a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:12.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.989+0000 7f68c175d640 1 -- 172.21.15.102:0/2593454900 >> 172.21.15.102:0/2593454900 conn(0x7f68bc09f8b0 msgr2=0x7f68bc09b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:12.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.989+0000 7f68c175d640 1 -- 172.21.15.102:0/2593454900 shutdown_connections 2024-09-13T17:42:12.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.989+0000 7f68c175d640 1 -- 172.21.15.102:0/2593454900 wait complete. 2024-09-13T17:42:12.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.991+0000 7f68c175d640 1 Processor -- start 2024-09-13T17:42:12.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.991+0000 7f68c175d640 1 -- start start 2024-09-13T17:42:12.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.991+0000 7f68c175d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f68bc097670 0x7f68bc090690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:12.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.991+0000 7f68c175d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68bc09ad50 0x7f68bc090bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:12.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.991+0000 7f68c175d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f68bc09b180 0x7f68bc091110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:12.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.991+0000 7f68baffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f68bc097670 0x7f68bc090690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:12.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.991+0000 7f68baffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f68bc097670 0x7f68bc090690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:46562/0 (socket says 172.21.15.102:46562) 2024-09-13T17:42:12.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.991+0000 7f68ba7fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68bc09ad50 0x7f68bc090bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:12.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.991+0000 7f68bb7fe640 1 --2- 172.21.15.102:0/98723513 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f68bc09b180 0x7f68bc091110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:12.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.992+0000 7f68baffd640 1 -- 172.21.15.102:0/98723513 learned_addr learned my addr 172.21.15.102:0/98723513 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:12.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.992+0000 7f68c175d640 1 -- 172.21.15.102:0/98723513 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68bc093d50 con 0x7f68bc09ad50 2024-09-13T17:42:12.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.992+0000 7f68c175d640 1 -- 172.21.15.102:0/98723513 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68bc0921c0 con 0x7f68bc09b180 2024-09-13T17:42:12.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.992+0000 7f68c175d640 1 -- 172.21.15.102:0/98723513 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f68bc09d950 con 0x7f68bc097670 2024-09-13T17:42:12.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.992+0000 7f68ba7fc640 1 -- 172.21.15.102:0/98723513 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f68bc097670 msgr2=0x7f68bc090690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:12.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.992+0000 7f68ba7fc640 1 --2- 172.21.15.102:0/98723513 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f68bc097670 0x7f68bc090690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:12.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.992+0000 7f68ba7fc640 1 -- 172.21.15.102:0/98723513 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f68bc09b180 msgr2=0x7f68bc091110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:12.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.992+0000 7f68ba7fc640 1 --2- 172.21.15.102:0/98723513 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f68bc09b180 0x7f68bc091110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:12.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.992+0000 7f68ba7fc640 1 -- 172.21.15.102:0/98723513 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b4009ae0 con 0x7f68bc09ad50 2024-09-13T17:42:12.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.992+0000 7f68bb7fe640 1 --2- 172.21.15.102:0/98723513 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f68bc09b180 0x7f68bc091110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:12.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.992+0000 7f68baffd640 1 --2- 172.21.15.102:0/98723513 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f68bc097670 0x7f68bc090690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:12.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.992+0000 7f68ba7fc640 1 --2- 172.21.15.102:0/98723513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68bc09ad50 0x7f68bc090bd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f68b000c490 tx=0x7f68b0007e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:12.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.992+0000 7f68a2ffd640 1 -- 172.21.15.102:0/98723513 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b00072e0 con 0x7f68bc09ad50 2024-09-13T17:42:12.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.992+0000 7f68c175d640 1 -- 172.21.15.102:0/98723513 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68bc05dd60 con 0x7f68bc09ad50 2024-09-13T17:42:12.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.992+0000 7f68a2ffd640 1 -- 172.21.15.102:0/98723513 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b0007460 con 0x7f68bc09ad50 2024-09-13T17:42:12.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.992+0000 7f68a2ffd640 1 -- 172.21.15.102:0/98723513 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b00047a0 con 0x7f68bc09ad50 2024-09-13T17:42:12.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.993+0000 7f68c175d640 1 -- 172.21.15.102:0/98723513 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68bc068de0 con 0x7f68bc09ad50 2024-09-13T17:42:12.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.994+0000 7f68a2ffd640 1 -- 172.21.15.102:0/98723513 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f68b0004bd0 con 0x7f68bc09ad50 2024-09-13T17:42:12.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.994+0000 7f68c175d640 1 -- 172.21.15.102:0/98723513 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68bc064600 con 0x7f68bc09ad50 2024-09-13T17:42:12.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:12.994+0000 7f68a2ffd640 1 -- 172.21.15.102:0/98723513 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7f68b00414a0 con 0x7f68bc09ad50 2024-09-13T17:42:13.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.002+0000 7f68a2ffd640 1 -- 172.21.15.102:0/98723513 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f68b000f680 con 0x7f68bc09ad50 2024-09-13T17:42:13.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.032+0000 7f359009f640 1 Processor -- start 2024-09-13T17:42:13.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.032+0000 7f359009f640 1 -- start start 2024-09-13T17:42:13.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.032+0000 7f359009f640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35880688b0 0x7f358809d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:13.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.032+0000 7f359009f640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f358809da90 0x7f358809de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:13.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.032+0000 7f359009f640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35880691f0 con 0x7f358812f4b0 2024-09-13T17:42:13.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.032+0000 7f359009f640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3588068c80 con 0x7f35880688b0 2024-09-13T17:42:13.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.033+0000 7f359009f640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f358809d360 con 0x7f358809da90 2024-09-13T17:42:13.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.033+0000 7f358d63a640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f358809da90 0x7f358809de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:13.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.033+0000 7f358d63a640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f358809da90 0x7f358809de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:46568/0 (socket says 172.21.15.102:46568) 2024-09-13T17:42:13.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.033+0000 7f358d63a640 1 -- 172.21.15.102:0/2908798388 learned_addr learned my addr 172.21.15.102:0/2908798388 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:13.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.033+0000 7f358e63c640 1 --1- 172.21.15.102:0/2908798388 >> v1:172.21.15.102:6789/0 conn(0x7f358812f4b0 0x7f358812f930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:60792/0 (socket says 172.21.15.102:60792) 2024-09-13T17:42:13.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.033+0000 7f358de3b640 1 --2- 172.21.15.102:0/2908798388 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35880688b0 0x7f358809d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:13.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.033+0000 7f358d63a640 1 -- 172.21.15.102:0/2908798388 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35880688b0 msgr2=0x7f358809d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:13.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.033+0000 7f358d63a640 1 --2- 172.21.15.102:0/2908798388 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35880688b0 0x7f358809d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.033+0000 7f358d63a640 1 -- 172.21.15.102:0/2908798388 >> v1:172.21.15.102:6789/0 conn(0x7f358812f4b0 legacy=0x7f358812f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:13.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.033+0000 7f358d63a640 1 -- 172.21.15.102:0/2908798388 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358809f090 con 0x7f358809da90 2024-09-13T17:42:13.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.034+0000 7f358d63a640 1 --2- 172.21.15.102:0/2908798388 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f358809da90 0x7f358809de80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3578009970 tx=0x7f35780091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=698a57834cc7095e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:13.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.034+0000 7f358ce39640 1 -- 172.21.15.102:0/2908798388 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357802ad00 con 0x7f358809da90 2024-09-13T17:42:13.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.034+0000 7f358ce39640 1 -- 172.21.15.102:0/2908798388 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f357802ae60 con 0x7f358809da90 2024-09-13T17:42:13.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.034+0000 7f358ce39640 1 -- 172.21.15.102:0/2908798388 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3578032670 con 0x7f358809da90 2024-09-13T17:42:13.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.035+0000 7f359009f640 1 -- 172.21.15.102:0/2908798388 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f358809da90 msgr2=0x7f358809de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:13.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.035+0000 7f359009f640 1 --2- 172.21.15.102:0/2908798388 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f358809da90 0x7f358809de80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3578009970 tx=0x7f35780091d0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.035+0000 7f359009f640 1 -- 172.21.15.102:0/2908798388 shutdown_connections 2024-09-13T17:42:13.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.035+0000 7f359009f640 1 --2- 172.21.15.102:0/2908798388 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f358809da90 0x7f358809de80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.035+0000 7f359009f640 1 --2- 172.21.15.102:0/2908798388 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35880688b0 0x7f358809d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.035+0000 7f359009f640 1 -- 172.21.15.102:0/2908798388 >> 172.21.15.102:0/2908798388 conn(0x7f358806f2e0 msgr2=0x7f35880a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:13.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.035+0000 7f359009f640 1 -- 172.21.15.102:0/2908798388 shutdown_connections 2024-09-13T17:42:13.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.035+0000 7f359009f640 1 -- 172.21.15.102:0/2908798388 wait complete. 2024-09-13T17:42:13.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.036+0000 7f359009f640 1 Processor -- start 2024-09-13T17:42:13.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.036+0000 7f359009f640 1 -- start start 2024-09-13T17:42:13.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.036+0000 7f359009f640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35880688b0 0x7f358811ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:13.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.036+0000 7f359009f640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f358809da90 0x7f358811b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:13.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.036+0000 7f358de3b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35880688b0 0x7f358811ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:13.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.036+0000 7f358de3b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35880688b0 0x7f358811ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35314/0 (socket says 172.21.15.102:35314) 2024-09-13T17:42:13.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.036+0000 7f359009f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f358812f4b0 0x7f358811b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:13.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.036+0000 7f358d63a640 1 --2- 172.21.15.102:0/218169783 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f358809da90 0x7f358811b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:13.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.036+0000 7f358de3b640 1 -- 172.21.15.102:0/218169783 learned_addr learned my addr 172.21.15.102:0/218169783 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:13.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.036+0000 7f359009f640 1 -- 172.21.15.102:0/218169783 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358811e190 con 0x7f358812f4b0 2024-09-13T17:42:13.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.036+0000 7f359009f640 1 -- 172.21.15.102:0/218169783 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f358809e7c0 con 0x7f35880688b0 2024-09-13T17:42:13.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.036+0000 7f359009f640 1 -- 172.21.15.102:0/218169783 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f358809fed0 con 0x7f358809da90 2024-09-13T17:42:13.040 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.036+0000 7f358e63c640 1 --2- 172.21.15.102:0/218169783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f358812f4b0 0x7f358811b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:13.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.037+0000 7f358de3b640 1 -- 172.21.15.102:0/218169783 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f358809da90 msgr2=0x7f358811b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:13.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.037+0000 7f358de3b640 1 --2- 172.21.15.102:0/218169783 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f358809da90 0x7f358811b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.037+0000 7f358de3b640 1 -- 172.21.15.102:0/218169783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f358812f4b0 msgr2=0x7f358811b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:13.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.037+0000 7f358de3b640 1 --2- 172.21.15.102:0/218169783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f358812f4b0 0x7f358811b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.037+0000 7f358de3b640 1 -- 172.21.15.102:0/218169783 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3578009450 con 0x7f35880688b0 2024-09-13T17:42:13.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.037+0000 7f358e63c640 1 --2- 172.21.15.102:0/218169783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f358812f4b0 0x7f358811b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:13.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.037+0000 7f358d63a640 1 --2- 172.21.15.102:0/218169783 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f358809da90 0x7f358811b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:13.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.037+0000 7f358de3b640 1 --2- 172.21.15.102:0/218169783 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35880688b0 0x7f358811ad20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f358400e900 tx=0x7f35840090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:13.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.037+0000 7f357dffb640 1 -- 172.21.15.102:0/218169783 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358400c8e0 con 0x7f35880688b0 2024-09-13T17:42:13.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.037+0000 7f357dffb640 1 -- 172.21.15.102:0/218169783 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f358400ca40 con 0x7f35880688b0 2024-09-13T17:42:13.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.037+0000 7f359009f640 1 -- 172.21.15.102:0/218169783 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f358811db80 con 0x7f35880688b0 2024-09-13T17:42:13.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.037+0000 7f357dffb640 1 -- 172.21.15.102:0/218169783 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358400cde0 con 0x7f35880688b0 2024-09-13T17:42:13.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.037+0000 7f359009f640 1 -- 172.21.15.102:0/218169783 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f358811c280 con 0x7f35880688b0 2024-09-13T17:42:13.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.038+0000 7f357dffb640 1 -- 172.21.15.102:0/218169783 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f3584013680 con 0x7f35880688b0 2024-09-13T17:42:13.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.039+0000 7f359009f640 1 -- 172.21.15.102:0/218169783 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f358811de90 con 0x7f35880688b0 2024-09-13T17:42:13.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.039+0000 7f357dffb640 1 -- 172.21.15.102:0/218169783 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7f35840425f0 con 0x7f35880688b0 2024-09-13T17:42:13.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.043+0000 7f357dffb640 1 -- 172.21.15.102:0/218169783 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f358401b770 con 0x7f35880688b0 2024-09-13T17:42:13.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.073+0000 7f4ed399d640 1 Processor -- start 2024-09-13T17:42:13.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.073+0000 7f4ed399d640 1 -- start start 2024-09-13T17:42:13.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.074+0000 7f4ed399d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4ecc099570 0x7f4ecc134720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:13.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.074+0000 7f4ed399d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4ecc134c60 0x7f4ecc137040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:13.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.074+0000 7f4ed399d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ecc09a7c0 con 0x7f4ecc09fff0 2024-09-13T17:42:13.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.074+0000 7f4ed399d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ecc0a03c0 con 0x7f4ecc099570 2024-09-13T17:42:13.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.074+0000 7f4ed399d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc09abb0 con 0x7f4ecc134c60 2024-09-13T17:42:13.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.074+0000 7f4ed0f38640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4ecc099570 0x7f4ecc134720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:13.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.074+0000 7f4ed1f3a640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4ecc134c60 0x7f4ecc137040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:13.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.074+0000 7f4ed0f38640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4ecc099570 0x7f4ecc134720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37368/0 (socket says 172.21.15.102:37368) 2024-09-13T17:42:13.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.074+0000 7f4ed0f38640 1 -- 172.21.15.102:0/3933558750 learned_addr learned my addr 172.21.15.102:0/3933558750 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:13.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.075+0000 7f4ed1f3a640 1 -- 172.21.15.102:0/3933558750 >> v2:172.21.15.102:3300/0 conn(0x7f4ecc099570 msgr2=0x7f4ecc134720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:13.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.075+0000 7f4ed1f3a640 1 --2- 172.21.15.102:0/3933558750 >> v2:172.21.15.102:3300/0 conn(0x7f4ecc099570 0x7f4ecc134720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.075+0000 7f4ed1f3a640 1 -- 172.21.15.102:0/3933558750 >> v1:172.21.15.102:6789/0 conn(0x7f4ecc09fff0 legacy=0x7f4ecc098e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:13.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.075+0000 7f4ec3fff640 1 -- 172.21.15.102:0/3933558750 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3449879881 0 0) 0x7f4ecc09a7c0 con 0x7f4ecc09fff0 2024-09-13T17:42:13.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.075+0000 7f4ed1f3a640 1 -- 172.21.15.102:0/3933558750 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ecc09bf70 con 0x7f4ecc134c60 2024-09-13T17:42:13.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.075+0000 7f4ed0f38640 1 --2- 172.21.15.102:0/3933558750 >> v2:172.21.15.102:3300/0 conn(0x7f4ecc099570 0x7f4ecc134720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:13.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.076+0000 7f4ed1f3a640 1 --2- 172.21.15.102:0/3933558750 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4ecc134c60 0x7f4ecc137040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009e60 tx=0x7f4ec4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=338664a72a5b456c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:13.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.076+0000 7f4ec3fff640 1 -- 172.21.15.102:0/3933558750 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4007550 con 0x7f4ecc134c60 2024-09-13T17:42:13.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.076+0000 7f4ec3fff640 1 -- 172.21.15.102:0/3933558750 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec402d050 con 0x7f4ecc134c60 2024-09-13T17:42:13.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.076+0000 7f4ec3fff640 1 -- 172.21.15.102:0/3933558750 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec40078e0 con 0x7f4ecc134c60 2024-09-13T17:42:13.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.077+0000 7f4ed399d640 1 -- 172.21.15.102:0/3933558750 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4ecc134c60 msgr2=0x7f4ecc137040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:13.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.077+0000 7f4ed399d640 1 --2- 172.21.15.102:0/3933558750 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4ecc134c60 0x7f4ecc137040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009e60 tx=0x7f4ec4009780 comp rx=0 tx=0).stop 2024-09-13T17:42:13.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.077+0000 7f4ed399d640 1 -- 172.21.15.102:0/3933558750 shutdown_connections 2024-09-13T17:42:13.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.077+0000 7f4ed399d640 1 --2- 172.21.15.102:0/3933558750 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4ecc134c60 0x7f4ecc137040 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.077+0000 7f4ed399d640 1 --2- 172.21.15.102:0/3933558750 >> v2:172.21.15.102:3300/0 conn(0x7f4ecc099570 0x7f4ecc134720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.077+0000 7f4ed399d640 1 -- 172.21.15.102:0/3933558750 >> 172.21.15.102:0/3933558750 conn(0x7f4ecc05e160 msgr2=0x7f4ecc09f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:13.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.077+0000 7f4ed399d640 1 -- 172.21.15.102:0/3933558750 shutdown_connections 2024-09-13T17:42:13.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.077+0000 7f4ed399d640 1 -- 172.21.15.102:0/3933558750 wait complete. 2024-09-13T17:42:13.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.078+0000 7f4ed399d640 1 Processor -- start 2024-09-13T17:42:13.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.078+0000 7f4ed399d640 1 -- start start 2024-09-13T17:42:13.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.078+0000 7f4ed399d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ecc099570 0x7f4ecc134170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:13.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.078+0000 7f4ed399d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4ecc09fff0 0x7f4ecc11e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:13.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.078+0000 7f4ed399d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4ecc134c60 0x7f4ecc11e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:13.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.079+0000 7f4ed399d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc1201f0 con 0x7f4ecc099570 2024-09-13T17:42:13.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.079+0000 7f4ed399d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc11fce0 con 0x7f4ecc09fff0 2024-09-13T17:42:13.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.079+0000 7f4ed399d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc121650 con 0x7f4ecc134c60 2024-09-13T17:42:13.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.079+0000 7f4ed1f3a640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4ecc134c60 0x7f4ecc11e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:13.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.079+0000 7f4ed1739640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ecc099570 0x7f4ecc134170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:13.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.079+0000 7f4ed1739640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ecc099570 0x7f4ecc134170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37372/0 (socket says 172.21.15.102:37372) 2024-09-13T17:42:13.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.079+0000 7f4ed1739640 1 -- 172.21.15.102:0/714601655 learned_addr learned my addr 172.21.15.102:0/714601655 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:13.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.079+0000 7f4ed1f3a640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4ecc134c60 0x7f4ecc11e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:46584/0 (socket says 172.21.15.102:46584) 2024-09-13T17:42:13.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.079+0000 7f4ed0f38640 1 --2- 172.21.15.102:0/714601655 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4ecc09fff0 0x7f4ecc11e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:13.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.079+0000 7f4ed1739640 1 -- 172.21.15.102:0/714601655 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4ecc134c60 msgr2=0x7f4ecc11e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:13.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.079+0000 7f4ed1739640 1 --2- 172.21.15.102:0/714601655 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4ecc134c60 0x7f4ecc11e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.079+0000 7f4ed1739640 1 -- 172.21.15.102:0/714601655 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4ecc09fff0 msgr2=0x7f4ecc11e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:13.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.079+0000 7f4ed1739640 1 --2- 172.21.15.102:0/714601655 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4ecc09fff0 0x7f4ecc11e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.079+0000 7f4ed1739640 1 -- 172.21.15.102:0/714601655 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ebc009450 con 0x7f4ecc099570 2024-09-13T17:42:13.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.079+0000 7f4ed1f3a640 1 --2- 172.21.15.102:0/714601655 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4ecc134c60 0x7f4ecc11e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:13.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.079+0000 7f4ed0f38640 1 --2- 172.21.15.102:0/714601655 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4ecc09fff0 0x7f4ecc11e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:13.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.079+0000 7f4ed1739640 1 --2- 172.21.15.102:0/714601655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ecc099570 0x7f4ecc134170 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80011e0 tx=0x7f4ec8007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:13.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.079+0000 7f4ec17fa640 1 -- 172.21.15.102:0/714601655 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8011590 con 0x7f4ecc099570 2024-09-13T17:42:13.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.080+0000 7f4ec17fa640 1 -- 172.21.15.102:0/714601655 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec80116f0 con 0x7f4ecc099570 2024-09-13T17:42:13.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.080+0000 7f4ed399d640 1 -- 172.21.15.102:0/714601655 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec4009a20 con 0x7f4ecc099570 2024-09-13T17:42:13.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.080+0000 7f4ec17fa640 1 -- 172.21.15.102:0/714601655 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8011ab0 con 0x7f4ecc099570 2024-09-13T17:42:13.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.080+0000 7f4ed399d640 1 -- 172.21.15.102:0/714601655 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ecc120460 con 0x7f4ecc099570 2024-09-13T17:42:13.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.081+0000 7f4ec17fa640 1 -- 172.21.15.102:0/714601655 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f4ec801b3b0 con 0x7f4ecc099570 2024-09-13T17:42:13.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.081+0000 7f4ec17fa640 1 -- 172.21.15.102:0/714601655 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7f4ec8041620 con 0x7f4ecc099570 2024-09-13T17:42:13.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.081+0000 7f4ed399d640 1 -- 172.21.15.102:0/714601655 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ecc064600 con 0x7f4ecc099570 2024-09-13T17:42:13.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.085+0000 7f4ec17fa640 1 -- 172.21.15.102:0/714601655 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f4ec8017030 con 0x7f4ecc099570 2024-09-13T17:42:13.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.104+0000 7f5f2551d640 1 Processor -- start 2024-09-13T17:42:13.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.104+0000 7f5f2551d640 1 -- start start 2024-09-13T17:42:13.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.105+0000 7f5f2551d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5f20068920 0x7f5f2009d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:13.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.105+0000 7f5f2551d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f2009e050 0x7f5f2012fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:13.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.105+0000 7f5f2551d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f200695a0 con 0x7f5f2009dc50 2024-09-13T17:42:13.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.105+0000 7f5f2551d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f20068cf0 con 0x7f5f20068920 2024-09-13T17:42:13.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.105+0000 7f5f2551d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2009e4a0 con 0x7f5f2009e050 2024-09-13T17:42:13.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.105+0000 7f5f1f7fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f2009e050 0x7f5f2012fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:13.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.105+0000 7f5f1f7fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f2009e050 0x7f5f2012fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35334/0 (socket says 172.21.15.102:35334) 2024-09-13T17:42:13.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.105+0000 7f5f1e7fc640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f5f2009dc50 0x7f5f2012f310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:60824/0 (socket says 172.21.15.102:60824) 2024-09-13T17:42:13.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.105+0000 7f5f1f7fe640 1 -- 172.21.15.102:0/424068878 learned_addr learned my addr 172.21.15.102:0/424068878 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:13.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.105+0000 7f5f1effd640 1 --2- 172.21.15.102:0/424068878 >> v2:172.21.15.102:3300/0 conn(0x7f5f20068920 0x7f5f2009d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:13.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.105+0000 7f5f1f7fe640 1 -- 172.21.15.102:0/424068878 >> v2:172.21.15.102:3300/0 conn(0x7f5f20068920 msgr2=0x7f5f2009d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:13.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.105+0000 7f5f1f7fe640 1 --2- 172.21.15.102:0/424068878 >> v2:172.21.15.102:3300/0 conn(0x7f5f20068920 0x7f5f2009d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.105+0000 7f5f1f7fe640 1 -- 172.21.15.102:0/424068878 >> v1:172.21.15.102:6789/0 conn(0x7f5f2009dc50 legacy=0x7f5f2012f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:13.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.105+0000 7f5f1f7fe640 1 -- 172.21.15.102:0/424068878 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2009f090 con 0x7f5f2009e050 2024-09-13T17:42:13.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.105+0000 7f5f1effd640 1 --2- 172.21.15.102:0/424068878 >> v2:172.21.15.102:3300/0 conn(0x7f5f20068920 0x7f5f2009d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:13.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.107+0000 7f5f1f7fe640 1 --2- 172.21.15.102:0/424068878 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f2009e050 0x7f5f2012fa20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5f18009f20 tx=0x7f5f18009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc7b2272fe3ca35c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:13.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.107+0000 7f5f1dffb640 1 -- 172.21.15.102:0/424068878 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f18031400 con 0x7f5f2009e050 2024-09-13T17:42:13.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.107+0000 7f5f1dffb640 1 -- 172.21.15.102:0/424068878 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1802d050 con 0x7f5f2009e050 2024-09-13T17:42:13.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.107+0000 7f5f1dffb640 1 -- 172.21.15.102:0/424068878 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f180317d0 con 0x7f5f2009e050 2024-09-13T17:42:13.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.108+0000 7f5f2551d640 1 -- 172.21.15.102:0/424068878 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f2009e050 msgr2=0x7f5f2012fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:13.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.108+0000 7f5f2551d640 1 --2- 172.21.15.102:0/424068878 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f2009e050 0x7f5f2012fa20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5f18009f20 tx=0x7f5f18009840 comp rx=0 tx=0).stop 2024-09-13T17:42:13.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.108+0000 7f5f2551d640 1 -- 172.21.15.102:0/424068878 shutdown_connections 2024-09-13T17:42:13.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.108+0000 7f5f2551d640 1 --2- 172.21.15.102:0/424068878 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f2009e050 0x7f5f2012fa20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.108+0000 7f5f2551d640 1 --2- 172.21.15.102:0/424068878 >> v2:172.21.15.102:3300/0 conn(0x7f5f20068920 0x7f5f2009d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.108+0000 7f5f2551d640 1 -- 172.21.15.102:0/424068878 >> 172.21.15.102:0/424068878 conn(0x7f5f2006f320 msgr2=0x7f5f200a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:13.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.108+0000 7f5f2551d640 1 -- 172.21.15.102:0/424068878 shutdown_connections 2024-09-13T17:42:13.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.108+0000 7f5f2551d640 1 -- 172.21.15.102:0/424068878 wait complete. 2024-09-13T17:42:13.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.109+0000 7f5f2551d640 1 Processor -- start 2024-09-13T17:42:13.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.109+0000 7f5f2551d640 1 -- start start 2024-09-13T17:42:13.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.109+0000 7f5f2551d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f20068920 0x7f5f2011f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:13.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.109+0000 7f5f2551d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f2009dc50 0x7f5f2011bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:13.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.109+0000 7f5f2551d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f2009e050 0x7f5f2011c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:13.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.110+0000 7f5f2551d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2009e620 con 0x7f5f2009e050 2024-09-13T17:42:13.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.110+0000 7f5f2551d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2011eb90 con 0x7f5f2009dc50 2024-09-13T17:42:13.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.110+0000 7f5f2551d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2011d300 con 0x7f5f20068920 2024-09-13T17:42:13.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.110+0000 7f5f1f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f2009e050 0x7f5f2011c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:13.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.110+0000 7f5f1e7fc640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f2009dc50 0x7f5f2011bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:13.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.110+0000 7f5f1effd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f20068920 0x7f5f2011f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:13.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.110+0000 7f5f1f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f2009e050 0x7f5f2011c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37384/0 (socket says 172.21.15.102:37384) 2024-09-13T17:42:13.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.110+0000 7f5f1f7fe640 1 -- 172.21.15.102:0/1466442697 learned_addr learned my addr 172.21.15.102:0/1466442697 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:13.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.110+0000 7f5f1f7fe640 1 -- 172.21.15.102:0/1466442697 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f20068920 msgr2=0x7f5f2011f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:13.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.110+0000 7f5f1f7fe640 1 --2- 172.21.15.102:0/1466442697 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f20068920 0x7f5f2011f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.110+0000 7f5f1f7fe640 1 -- 172.21.15.102:0/1466442697 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f2009dc50 msgr2=0x7f5f2011bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:13.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.110+0000 7f5f1f7fe640 1 --2- 172.21.15.102:0/1466442697 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f2009dc50 0x7f5f2011bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.110+0000 7f5f1f7fe640 1 -- 172.21.15.102:0/1466442697 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f10009390 con 0x7f5f2009e050 2024-09-13T17:42:13.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.110+0000 7f5f1effd640 1 --2- 172.21.15.102:0/1466442697 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f20068920 0x7f5f2011f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:13.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.110+0000 7f5f1e7fc640 1 --2- 172.21.15.102:0/1466442697 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f2009dc50 0x7f5f2011bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:13.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.110+0000 7f5f1f7fe640 1 --2- 172.21.15.102:0/1466442697 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f2009e050 0x7f5f2011c4e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5f1802ab50 tx=0x7f5f18036000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:13.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.111+0000 7f5f06ffd640 1 -- 172.21.15.102:0/1466442697 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f18007170 con 0x7f5f2009e050 2024-09-13T17:42:13.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.111+0000 7f5f06ffd640 1 -- 172.21.15.102:0/1466442697 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f18007e40 con 0x7f5f2009e050 2024-09-13T17:42:13.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.111+0000 7f5f06ffd640 1 -- 172.21.15.102:0/1466442697 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1803c670 con 0x7f5f2009e050 2024-09-13T17:42:13.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.111+0000 7f5f2551d640 1 -- 172.21.15.102:0/1466442697 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f18009ae0 con 0x7f5f2009e050 2024-09-13T17:42:13.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.111+0000 7f5f2551d640 1 -- 172.21.15.102:0/1466442697 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f2011bbb0 con 0x7f5f2009e050 2024-09-13T17:42:13.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.112+0000 7f5f06ffd640 1 -- 172.21.15.102:0/1466442697 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f5f1803ca60 con 0x7f5f2009e050 2024-09-13T17:42:13.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.112+0000 7f5f2551d640 1 -- 172.21.15.102:0/1466442697 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f20098200 con 0x7f5f2009e050 2024-09-13T17:42:13.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.112+0000 7f5f06ffd640 1 -- 172.21.15.102:0/1466442697 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7f5f18062bd0 con 0x7f5f2009e050 2024-09-13T17:42:13.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.117+0000 7f5f06ffd640 1 -- 172.21.15.102:0/1466442697 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f5f20098200 con 0x7f5f2009e050 2024-09-13T17:42:13.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.204+0000 7f68c175d640 1 -- 172.21.15.102:0/98723513 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f68bc093e90 con 0x7f68bc09ad50 2024-09-13T17:42:13.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.204+0000 7f68a2ffd640 1 -- 172.21.15.102:0/98723513 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f68bc093e90 con 0x7f68bc09ad50 2024-09-13T17:42:13.207 INFO:teuthology.orchestra.run.smithi102.stdout:38654705671 2024-09-13T17:42:13.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.207+0000 7f359009f640 1 -- 172.21.15.102:0/218169783 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3588064000 con 0x7f35880688b0 2024-09-13T17:42:13.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.207+0000 7f357dffb640 1 -- 172.21.15.102:0/218169783 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f358401b3b0 con 0x7f35880688b0 2024-09-13T17:42:13.209 INFO:teuthology.orchestra.run.smithi102.stdout:38654705669 2024-09-13T17:42:13.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.207+0000 7f68c175d640 1 -- 172.21.15.102:0/98723513 >> v1:172.21.15.102:6816/34937 conn(0x7f68a8031160 legacy=0x7f68a8033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:13.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.207+0000 7f68c175d640 1 -- 172.21.15.102:0/98723513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68bc09ad50 msgr2=0x7f68bc090bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:13.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.207+0000 7f68c175d640 1 --2- 172.21.15.102:0/98723513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68bc09ad50 0x7f68bc090bd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f68b000c490 tx=0x7f68b0007e20 comp rx=0 tx=0).stop 2024-09-13T17:42:13.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.208+0000 7f68c175d640 1 -- 172.21.15.102:0/98723513 shutdown_connections 2024-09-13T17:42:13.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.208+0000 7f68c175d640 1 --2- 172.21.15.102:0/98723513 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f68bc09b180 0x7f68bc091110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.208+0000 7f68c175d640 1 --2- 172.21.15.102:0/98723513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68bc09ad50 0x7f68bc090bd0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.208+0000 7f68c175d640 1 --2- 172.21.15.102:0/98723513 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f68bc097670 0x7f68bc090690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.208+0000 7f68c175d640 1 -- 172.21.15.102:0/98723513 >> 172.21.15.102:0/98723513 conn(0x7f68bc09f8b0 msgr2=0x7f68bc099800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:13.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.208+0000 7f68c175d640 1 -- 172.21.15.102:0/98723513 shutdown_connections 2024-09-13T17:42:13.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.208+0000 7f68c175d640 1 -- 172.21.15.102:0/98723513 wait complete. 2024-09-13T17:42:13.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.209+0000 7f359009f640 1 -- 172.21.15.102:0/218169783 >> v1:172.21.15.102:6816/34937 conn(0x7f356c031190 legacy=0x7f356c033640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:13.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.210+0000 7f359009f640 1 -- 172.21.15.102:0/218169783 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35880688b0 msgr2=0x7f358811ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:13.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.210+0000 7f359009f640 1 --2- 172.21.15.102:0/218169783 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35880688b0 0x7f358811ad20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f358400e900 tx=0x7f35840090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.210+0000 7f359009f640 1 -- 172.21.15.102:0/218169783 shutdown_connections 2024-09-13T17:42:13.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.210+0000 7f359009f640 1 --2- 172.21.15.102:0/218169783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f358812f4b0 0x7f358811b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.210+0000 7f359009f640 1 --2- 172.21.15.102:0/218169783 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f358809da90 0x7f358811b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.210+0000 7f359009f640 1 --2- 172.21.15.102:0/218169783 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f35880688b0 0x7f358811ad20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.210+0000 7f359009f640 1 -- 172.21.15.102:0/218169783 >> 172.21.15.102:0/218169783 conn(0x7f358806f2e0 msgr2=0x7f35880a1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:13.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.210+0000 7f359009f640 1 -- 172.21.15.102:0/218169783 shutdown_connections 2024-09-13T17:42:13.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.210+0000 7f359009f640 1 -- 172.21.15.102:0/218169783 wait complete. 2024-09-13T17:42:13.221 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705672 got 38654705671 for osd.1 2024-09-13T17:42:13.227 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705669 for osd.0 2024-09-13T17:42:13.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.269+0000 7f4ed399d640 1 -- 172.21.15.102:0/714601655 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4ecc09b510 con 0x7f4ecc099570 2024-09-13T17:42:13.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.269+0000 7f4ec17fa640 1 -- 172.21.15.102:0/714601655 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4ec8017020 con 0x7f4ecc099570 2024-09-13T17:42:13.271 INFO:teuthology.orchestra.run.smithi102.stdout:38654705668 2024-09-13T17:42:13.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.271+0000 7f4ed399d640 1 -- 172.21.15.102:0/714601655 >> v1:172.21.15.102:6816/34937 conn(0x7f4eb0031160 legacy=0x7f4eb0033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:13.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.271+0000 7f4ed399d640 1 -- 172.21.15.102:0/714601655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ecc099570 msgr2=0x7f4ecc134170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:13.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.271+0000 7f4ed399d640 1 --2- 172.21.15.102:0/714601655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ecc099570 0x7f4ecc134170 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80011e0 tx=0x7f4ec8007d10 comp rx=0 tx=0).stop 2024-09-13T17:42:13.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.271+0000 7f4ed399d640 1 -- 172.21.15.102:0/714601655 shutdown_connections 2024-09-13T17:42:13.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.271+0000 7f4ed399d640 1 --2- 172.21.15.102:0/714601655 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4ecc134c60 0x7f4ecc11e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.271+0000 7f4ed399d640 1 --2- 172.21.15.102:0/714601655 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4ecc09fff0 0x7f4ecc11e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.271+0000 7f4ed399d640 1 --2- 172.21.15.102:0/714601655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ecc099570 0x7f4ecc134170 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.271+0000 7f4ed399d640 1 -- 172.21.15.102:0/714601655 >> 172.21.15.102:0/714601655 conn(0x7f4ecc05e160 msgr2=0x7f4ecc06e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:13.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.272+0000 7f4ed399d640 1 -- 172.21.15.102:0/714601655 shutdown_connections 2024-09-13T17:42:13.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.272+0000 7f4ed399d640 1 -- 172.21.15.102:0/714601655 wait complete. 2024-09-13T17:42:13.284 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705668 for osd.3 2024-09-13T17:42:13.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.298+0000 7f5f2551d640 1 -- 172.21.15.102:0/1466442697 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5f20098200 con 0x7f5f2009e050 2024-09-13T17:42:13.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.298+0000 7f5f06ffd640 1 -- 172.21.15.102:0/1466442697 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5f20098200 con 0x7f5f2009e050 2024-09-13T17:42:13.300 INFO:teuthology.orchestra.run.smithi102.stdout:38654705668 2024-09-13T17:42:13.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.300+0000 7f5f2551d640 1 -- 172.21.15.102:0/1466442697 >> v1:172.21.15.102:6816/34937 conn(0x7f5f0c031160 legacy=0x7f5f0c033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:13.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.300+0000 7f5f2551d640 1 -- 172.21.15.102:0/1466442697 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f2009e050 msgr2=0x7f5f2011c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:13.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.300+0000 7f5f2551d640 1 --2- 172.21.15.102:0/1466442697 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f2009e050 0x7f5f2011c4e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5f1802ab50 tx=0x7f5f18036000 comp rx=0 tx=0).stop 2024-09-13T17:42:13.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.300+0000 7f5f2551d640 1 -- 172.21.15.102:0/1466442697 shutdown_connections 2024-09-13T17:42:13.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.300+0000 7f5f2551d640 1 --2- 172.21.15.102:0/1466442697 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f2009e050 0x7f5f2011c4e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.300+0000 7f5f2551d640 1 --2- 172.21.15.102:0/1466442697 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f2009dc50 0x7f5f2011bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.300+0000 7f5f2551d640 1 --2- 172.21.15.102:0/1466442697 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f20068920 0x7f5f2011f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.300+0000 7f5f2551d640 1 -- 172.21.15.102:0/1466442697 >> 172.21.15.102:0/1466442697 conn(0x7f5f2006f320 msgr2=0x7f5f20098440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:13.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.300+0000 7f5f2551d640 1 -- 172.21.15.102:0/1466442697 shutdown_connections 2024-09-13T17:42:13.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:13.300+0000 7f5f2551d640 1 -- 172.21.15.102:0/1466442697 wait complete. 2024-09-13T17:42:13.312 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705668 for osd.2 2024-09-13T17:42:14.222 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T17:42:14.228 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T17:42:14.285 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T17:42:14.313 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T17:42:14.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.340+0000 7f32cd88b640 1 Processor -- start 2024-09-13T17:42:14.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.340+0000 7f32cd88b640 1 -- start start 2024-09-13T17:42:14.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.341+0000 7f32cd88b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32c809af20 0x7f32c8134fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.341+0000 7f32cd88b640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f32c809b350 0x7f32c8137520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.341+0000 7f32cd88b640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32c8095970 con 0x7f32c8097670 2024-09-13T17:42:14.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.341+0000 7f32cd88b640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32c80971c0 con 0x7f32c809b350 2024-09-13T17:42:14.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.341+0000 7f32cd88b640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c8097a40 con 0x7f32c809af20 2024-09-13T17:42:14.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.341+0000 7f32c77fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f32c809b350 0x7f32c8137520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.341+0000 7f32bffff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32c809af20 0x7f32c8134fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.341+0000 7f32c77fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f32c809b350 0x7f32c8137520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37400/0 (socket says 172.21.15.102:37400) 2024-09-13T17:42:14.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.341+0000 7f32c77fe640 1 -- 172.21.15.102:0/2070843897 learned_addr learned my addr 172.21.15.102:0/2070843897 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:14.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.341+0000 7f32c6ffd640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f32c8097670 0x7f32c809a810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:60840/0 (socket says 172.21.15.102:60840) 2024-09-13T17:42:14.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.341+0000 7f32bffff640 1 -- 172.21.15.102:0/2070843897 >> v2:172.21.15.102:3300/0 conn(0x7f32c809b350 msgr2=0x7f32c8137520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:14.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.341+0000 7f32bffff640 1 --2- 172.21.15.102:0/2070843897 >> v2:172.21.15.102:3300/0 conn(0x7f32c809b350 0x7f32c8137520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.341+0000 7f32bffff640 1 -- 172.21.15.102:0/2070843897 >> v1:172.21.15.102:6789/0 conn(0x7f32c8097670 legacy=0x7f32c809a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.341+0000 7f32bffff640 1 -- 172.21.15.102:0/2070843897 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c8096960 con 0x7f32c809af20 2024-09-13T17:42:14.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.342+0000 7f32bffff640 1 --2- 172.21.15.102:0/2070843897 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32c809af20 0x7f32c8134fd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009850 tx=0x7f32b00090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d700d1e391b7ab3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:14.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.342+0000 7f32c67fc640 1 -- 172.21.15.102:0/2070843897 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b002ac80 con 0x7f32c809af20 2024-09-13T17:42:14.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.343+0000 7f32c67fc640 1 -- 172.21.15.102:0/2070843897 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b002ade0 con 0x7f32c809af20 2024-09-13T17:42:14.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.343+0000 7f32c67fc640 1 -- 172.21.15.102:0/2070843897 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b0032670 con 0x7f32c809af20 2024-09-13T17:42:14.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.343+0000 7f32cd88b640 1 -- 172.21.15.102:0/2070843897 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32c809af20 msgr2=0x7f32c8134fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.343+0000 7f32cd88b640 1 --2- 172.21.15.102:0/2070843897 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32c809af20 0x7f32c8134fd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009850 tx=0x7f32b00090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.344+0000 7f32cd88b640 1 -- 172.21.15.102:0/2070843897 shutdown_connections 2024-09-13T17:42:14.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.344+0000 7f32cd88b640 1 --2- 172.21.15.102:0/2070843897 >> v2:172.21.15.102:3300/0 conn(0x7f32c809b350 0x7f32c8137520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.344+0000 7f32cd88b640 1 --2- 172.21.15.102:0/2070843897 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32c809af20 0x7f32c8134fd0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.344+0000 7f32cd88b640 1 -- 172.21.15.102:0/2070843897 >> 172.21.15.102:0/2070843897 conn(0x7f32c809f8b0 msgr2=0x7f32c809b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:14.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.344+0000 7f32cd88b640 1 -- 172.21.15.102:0/2070843897 shutdown_connections 2024-09-13T17:42:14.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.344+0000 7f32cd88b640 1 -- 172.21.15.102:0/2070843897 wait complete. 2024-09-13T17:42:14.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.344+0000 7f32cd88b640 1 Processor -- start 2024-09-13T17:42:14.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.345+0000 7f32cd88b640 1 -- start start 2024-09-13T17:42:14.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.345+0000 7f32cd88b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32c8097670 0x7f32c8094740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.345+0000 7f32cd88b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32c809af20 0x7f32c8090ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.345+0000 7f32cd88b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32c809b350 0x7f32c8091000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.345+0000 7f32cd88b640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c809dcc0 con 0x7f32c809b350 2024-09-13T17:42:14.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.345+0000 7f32cd88b640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c8096520 con 0x7f32c8097670 2024-09-13T17:42:14.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.345+0000 7f32cd88b640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32c8092450 con 0x7f32c809af20 2024-09-13T17:42:14.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.345+0000 7f32c77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32c809b350 0x7f32c8091000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.345+0000 7f32c77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32c809b350 0x7f32c8091000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37404/0 (socket says 172.21.15.102:37404) 2024-09-13T17:42:14.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.345+0000 7f32c77fe640 1 -- 172.21.15.102:0/3214981790 learned_addr learned my addr 172.21.15.102:0/3214981790 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:14.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.345+0000 7f32bffff640 1 --2- 172.21.15.102:0/3214981790 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32c809af20 0x7f32c8090ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.346+0000 7f32c77fe640 1 -- 172.21.15.102:0/3214981790 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32c809af20 msgr2=0x7f32c8090ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.346+0000 7f32c77fe640 1 --2- 172.21.15.102:0/3214981790 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32c809af20 0x7f32c8090ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.346+0000 7f32c77fe640 1 -- 172.21.15.102:0/3214981790 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32c8097670 msgr2=0x7f32c8094740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.345+0000 7f32c6ffd640 1 --2- 172.21.15.102:0/3214981790 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32c8097670 0x7f32c8094740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.346+0000 7f32c77fe640 1 --2- 172.21.15.102:0/3214981790 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32c8097670 0x7f32c8094740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.346+0000 7f32c77fe640 1 -- 172.21.15.102:0/3214981790 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b0009390 con 0x7f32c809b350 2024-09-13T17:42:14.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.346+0000 7f32bffff640 1 --2- 172.21.15.102:0/3214981790 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32c809af20 0x7f32c8090ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:14.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.346+0000 7f32c6ffd640 1 --2- 172.21.15.102:0/3214981790 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32c8097670 0x7f32c8094740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:42:14.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.346+0000 7f32c77fe640 1 --2- 172.21.15.102:0/3214981790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32c809b350 0x7f32c8091000 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009e50 tx=0x7f32c000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:14.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.346+0000 7f32beffd640 1 -- 172.21.15.102:0/3214981790 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c000cb30 con 0x7f32c809b350 2024-09-13T17:42:14.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.346+0000 7f32beffd640 1 -- 172.21.15.102:0/3214981790 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c000f040 con 0x7f32c809b350 2024-09-13T17:42:14.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.346+0000 7f32cd88b640 1 -- 172.21.15.102:0/3214981790 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c805dd60 con 0x7f32c809b350 2024-09-13T17:42:14.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.346+0000 7f32beffd640 1 -- 172.21.15.102:0/3214981790 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0014440 con 0x7f32c809b350 2024-09-13T17:42:14.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.346+0000 7f32cd88b640 1 -- 172.21.15.102:0/3214981790 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c8068de0 con 0x7f32c809b350 2024-09-13T17:42:14.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.347+0000 7f32beffd640 1 -- 172.21.15.102:0/3214981790 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f32c00147f0 con 0x7f32c809b350 2024-09-13T17:42:14.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.348+0000 7f32beffd640 1 -- 172.21.15.102:0/3214981790 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7f32c0042780 con 0x7f32c809b350 2024-09-13T17:42:14.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.348+0000 7f32cd88b640 1 -- 172.21.15.102:0/3214981790 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c8064600 con 0x7f32c809b350 2024-09-13T17:42:14.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.354+0000 7f32beffd640 1 -- 172.21.15.102:0/3214981790 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f32c001d400 con 0x7f32c809b350 2024-09-13T17:42:14.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.383+0000 7ffb2f0c0640 1 Processor -- start 2024-09-13T17:42:14.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.383+0000 7ffb2f0c0640 1 -- start start 2024-09-13T17:42:14.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.383+0000 7ffb2f0c0640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ffb2809de20 0x7ffb28134680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.383+0000 7ffb2f0c0640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffb2809e220 0x7ffb28136bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.383+0000 7ffb2f0c0640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb280695a0 con 0x7ffb28068920 2024-09-13T17:42:14.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.383+0000 7ffb2f0c0640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb28068cf0 con 0x7ffb2809de20 2024-09-13T17:42:14.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.383+0000 7ffb27fff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ffb2809de20 0x7ffb28134680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.383+0000 7ffb2ce5c640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7ffb28068920 0x7ffb2809d710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:60856/0 (socket says 172.21.15.102:60856) 2024-09-13T17:42:14.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.383+0000 7ffb2f0c0640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2809f090 con 0x7ffb2809e220 2024-09-13T17:42:14.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.383+0000 7ffb2ce5c640 1 -- 172.21.15.102:0/2740079038 learned_addr learned my addr 172.21.15.102:0/2740079038 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:42:14.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.383+0000 7ffb2d65d640 1 --2- 172.21.15.102:0/2740079038 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffb2809e220 0x7ffb28136bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.384+0000 7ffb27fff640 1 -- 172.21.15.102:0/2740079038 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffb2809e220 msgr2=0x7ffb28136bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:14.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.384+0000 7ffb27fff640 1 --2- 172.21.15.102:0/2740079038 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffb2809e220 0x7ffb28136bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.384+0000 7ffb27fff640 1 -- 172.21.15.102:0/2740079038 >> v1:172.21.15.102:6789/0 conn(0x7ffb28068920 legacy=0x7ffb2809d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:14.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.384+0000 7ffb27fff640 1 -- 172.21.15.102:0/2740079038 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2809cce0 con 0x7ffb2809de20 2024-09-13T17:42:14.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.385+0000 7ffb27fff640 1 --2- 172.21.15.102:0/2740079038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb2809de20 0x7ffb28134680 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffb14009850 tx=0x7ffb140090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b0b03faf440a67 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:14.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.385+0000 7ffb277fe640 1 -- 172.21.15.102:0/2740079038 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1402abe0 con 0x7ffb2809de20 2024-09-13T17:42:14.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.385+0000 7ffb277fe640 1 -- 172.21.15.102:0/2740079038 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb1402ad40 con 0x7ffb2809de20 2024-09-13T17:42:14.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.385+0000 7ffb277fe640 1 -- 172.21.15.102:0/2740079038 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb14032670 con 0x7ffb2809de20 2024-09-13T17:42:14.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.385+0000 7ffb2f0c0640 1 -- 172.21.15.102:0/2740079038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb2809de20 msgr2=0x7ffb28134680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.385+0000 7ffb2f0c0640 1 --2- 172.21.15.102:0/2740079038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb2809de20 0x7ffb28134680 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffb14009850 tx=0x7ffb140090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.386+0000 7ffb2f0c0640 1 -- 172.21.15.102:0/2740079038 shutdown_connections 2024-09-13T17:42:14.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.386+0000 7ffb2f0c0640 1 --2- 172.21.15.102:0/2740079038 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffb2809e220 0x7ffb28136bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.386+0000 7ffb2f0c0640 1 --2- 172.21.15.102:0/2740079038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb2809de20 0x7ffb28134680 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.386+0000 7ffb2f0c0640 1 -- 172.21.15.102:0/2740079038 >> 172.21.15.102:0/2740079038 conn(0x7ffb2806f320 msgr2=0x7ffb280a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:14.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.386+0000 7ffb2f0c0640 1 -- 172.21.15.102:0/2740079038 shutdown_connections 2024-09-13T17:42:14.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.386+0000 7ffb2f0c0640 1 -- 172.21.15.102:0/2740079038 wait complete. 2024-09-13T17:42:14.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.386+0000 7ffb2f0c0640 1 Processor -- start 2024-09-13T17:42:14.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.387+0000 7ffb2f0c0640 1 -- start start 2024-09-13T17:42:14.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.387+0000 7ffb2f0c0640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffb28068920 0x7ffb28134210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.387+0000 7ffb2f0c0640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffb2809de20 0x7ffb28134750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.387+0000 7ffb2f0c0640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb28119ce0 0x7ffb28134c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.387+0000 7ffb2ce5c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffb28068920 0x7ffb28134210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.387+0000 7ffb27fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffb2809de20 0x7ffb28134750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.387+0000 7ffb2ce5c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffb28068920 0x7ffb28134210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35386/0 (socket says 172.21.15.102:35386) 2024-09-13T17:42:14.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.387+0000 7ffb27fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffb2809de20 0x7ffb28134750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:46624/0 (socket says 172.21.15.102:46624) 2024-09-13T17:42:14.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.387+0000 7ffb2ce5c640 1 -- 172.21.15.102:0/4096352822 learned_addr learned my addr 172.21.15.102:0/4096352822 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:14.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.387+0000 7ffb2d65d640 1 --2- 172.21.15.102:0/4096352822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb28119ce0 0x7ffb28134c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.387+0000 7ffb2f0c0640 1 -- 172.21.15.102:0/4096352822 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb28136ca0 con 0x7ffb28119ce0 2024-09-13T17:42:14.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.387+0000 7ffb2f0c0640 1 -- 172.21.15.102:0/4096352822 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2809e620 con 0x7ffb28068920 2024-09-13T17:42:14.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.387+0000 7ffb2f0c0640 1 -- 172.21.15.102:0/4096352822 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2811a630 con 0x7ffb2809de20 2024-09-13T17:42:14.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.388+0000 7ffb27fff640 1 -- 172.21.15.102:0/4096352822 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffb28068920 msgr2=0x7ffb28134210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.388+0000 7ffb27fff640 1 --2- 172.21.15.102:0/4096352822 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffb28068920 0x7ffb28134210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.388+0000 7ffb27fff640 1 -- 172.21.15.102:0/4096352822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb28119ce0 msgr2=0x7ffb28134c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.388+0000 7ffb27fff640 1 --2- 172.21.15.102:0/4096352822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb28119ce0 0x7ffb28134c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.388+0000 7ffb27fff640 1 -- 172.21.15.102:0/4096352822 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb14009390 con 0x7ffb2809de20 2024-09-13T17:42:14.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.388+0000 7ffb2d65d640 1 --2- 172.21.15.102:0/4096352822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb28119ce0 0x7ffb28134c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:14.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.388+0000 7ffb2ce5c640 1 --2- 172.21.15.102:0/4096352822 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffb28068920 0x7ffb28134210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:14.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.388+0000 7ffb27fff640 1 --2- 172.21.15.102:0/4096352822 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffb2809de20 0x7ffb28134750 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ffb28069730 tx=0x7ffb14032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:14.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.388+0000 7ffb24ff9640 1 -- 172.21.15.102:0/4096352822 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb14031780 con 0x7ffb2809de20 2024-09-13T17:42:14.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.388+0000 7ffb24ff9640 1 -- 172.21.15.102:0/4096352822 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb14031920 con 0x7ffb2809de20 2024-09-13T17:42:14.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.388+0000 7ffb24ff9640 1 -- 172.21.15.102:0/4096352822 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb14031ce0 con 0x7ffb2809de20 2024-09-13T17:42:14.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.388+0000 7ffb2f0c0640 1 -- 172.21.15.102:0/4096352822 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb2811b040 con 0x7ffb2809de20 2024-09-13T17:42:14.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.388+0000 7ffb2f0c0640 1 -- 172.21.15.102:0/4096352822 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb2811a370 con 0x7ffb2809de20 2024-09-13T17:42:14.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.389+0000 7ffb24ff9640 1 -- 172.21.15.102:0/4096352822 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7ffb14039040 con 0x7ffb2809de20 2024-09-13T17:42:14.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.390+0000 7ffb24ff9640 1 -- 172.21.15.102:0/4096352822 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7ffb14061a50 con 0x7ffb2809de20 2024-09-13T17:42:14.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.390+0000 7ffb2f0c0640 1 -- 172.21.15.102:0/4096352822 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb28064000 con 0x7ffb2809de20 2024-09-13T17:42:14.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.398+0000 7ffb24ff9640 1 -- 172.21.15.102:0/4096352822 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7ffb14042020 con 0x7ffb2809de20 2024-09-13T17:42:14.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.450+0000 7f739719d640 1 Processor -- start 2024-09-13T17:42:14.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.450+0000 7f739719d640 1 -- start start 2024-09-13T17:42:14.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.450+0000 7f6f2d79d640 1 Processor -- start 2024-09-13T17:42:14.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.450+0000 7f6f2d79d640 1 -- start start 2024-09-13T17:42:14.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.451+0000 7f739719d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f739006dca0 0x7f739006d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.451+0000 7f739719d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f739012d8b0 0x7f7390135530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.451+0000 7f739719d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f739009ff20 con 0x7f739009f7f0 2024-09-13T17:42:14.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.451+0000 7f739719d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f739006e070 con 0x7f739012d8b0 2024-09-13T17:42:14.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.451+0000 7f738ffff640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f739009f7f0 0x7f739012d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:60862/0 (socket says 172.21.15.102:60862) 2024-09-13T17:42:14.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.451+0000 7f739719d640 1 -- 172.21.15.102:0/1690485924 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f739009fbc0 con 0x7f739006dca0 2024-09-13T17:42:14.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.451+0000 7f738ffff640 1 -- 172.21.15.102:0/1690485924 learned_addr learned my addr 172.21.15.102:0/1690485924 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:42:14.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.451+0000 7f6f2d79d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6f2809af20 0x7f6f28130f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.451+0000 7f7394f39640 1 --2- 172.21.15.102:0/1690485924 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f739006dca0 0x7f739006d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.451+0000 7f739573a640 1 --2- 172.21.15.102:0/1690485924 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f739012d8b0 0x7f7390135530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.451+0000 7f6f2d79d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6f2809b320 0x7f6f281334e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.451+0000 7f6f267fc640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6f2809af20 0x7f6f28130f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.451+0000 7f6f26ffd640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f6f2809c2a0 0x7f6f2809a810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:60870/0 (socket says 172.21.15.102:60870) 2024-09-13T17:42:14.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.451+0000 7f6f26ffd640 1 -- 172.21.15.102:0/77819144 learned_addr learned my addr 172.21.15.102:0/77819144 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:42:14.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.451+0000 7f738f7fe640 1 -- 172.21.15.102:0/1690485924 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3660477920 0 0) 0x7f739009ff20 con 0x7f739009f7f0 2024-09-13T17:42:14.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.451+0000 7f738f7fe640 1 -- 172.21.15.102:0/1690485924 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73800034e0 con 0x7f739009f7f0 2024-09-13T17:42:14.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.451+0000 7f6f277fe640 1 --2- 172.21.15.102:0/77819144 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6f2809b320 0x7f6f281334e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.451+0000 7f6f2d79d640 1 -- 172.21.15.102:0/77819144 --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f28096710 con 0x7f6f2809c2a0 2024-09-13T17:42:14.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.451+0000 7f6f2d79d640 1 -- 172.21.15.102:0/77819144 --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f2809c670 con 0x7f6f2809af20 2024-09-13T17:42:14.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.451+0000 7f6f2d79d640 1 -- 172.21.15.102:0/77819144 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f28097440 con 0x7f6f2809b320 2024-09-13T17:42:14.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.452+0000 7f739573a640 1 -- 172.21.15.102:0/1690485924 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f739006dca0 msgr2=0x7f739006d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:14.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.452+0000 7f739573a640 1 --2- 172.21.15.102:0/1690485924 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f739006dca0 0x7f739006d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.452+0000 7f739573a640 1 -- 172.21.15.102:0/1690485924 >> v1:172.21.15.102:6789/0 conn(0x7f739009f7f0 legacy=0x7f739012d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.452+0000 7f6f25ffb640 1 -- 172.21.15.102:0/77819144 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 580295635 0 0) 0x7f6f28096710 con 0x7f6f2809c2a0 2024-09-13T17:42:14.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.452+0000 7f6f25ffb640 1 -- 172.21.15.102:0/77819144 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f100034e0 con 0x7f6f2809c2a0 2024-09-13T17:42:14.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.452+0000 7f738f7fe640 1 -- 172.21.15.102:0/1690485924 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2058238841 0 0) 0x7f73800034e0 con 0x7f739009f7f0 2024-09-13T17:42:14.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.452+0000 7f739573a640 1 -- 172.21.15.102:0/1690485924 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f739006e990 con 0x7f739012d8b0 2024-09-13T17:42:14.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.452+0000 7f6f267fc640 1 -- 172.21.15.102:0/77819144 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6f2809b320 msgr2=0x7f6f281334e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:14.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.452+0000 7f6f267fc640 1 --2- 172.21.15.102:0/77819144 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6f2809b320 0x7f6f281334e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.452+0000 7f6f267fc640 1 -- 172.21.15.102:0/77819144 >> v1:172.21.15.102:6789/0 conn(0x7f6f2809c2a0 legacy=0x7f6f2809a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.452+0000 7f6f267fc640 1 -- 172.21.15.102:0/77819144 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f28098770 con 0x7f6f2809af20 2024-09-13T17:42:14.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.453+0000 7f739573a640 1 --2- 172.21.15.102:0/1690485924 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f739012d8b0 0x7f7390135530 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7388009ec0 tx=0x7f7388009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f8a3cdd1b1cbb99 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:14.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.453+0000 7f738f7fe640 1 -- 172.21.15.102:0/1690485924 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c001470 con 0x7f739012d8b0 2024-09-13T17:42:14.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.453+0000 7f738f7fe640 1 -- 172.21.15.102:0/1690485924 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f738802d050 con 0x7f739012d8b0 2024-09-13T17:42:14.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.453+0000 7f738f7fe640 1 -- 172.21.15.102:0/1690485924 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7388031630 con 0x7f739012d8b0 2024-09-13T17:42:14.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.453+0000 7f6f267fc640 1 --2- 172.21.15.102:0/77819144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f2809af20 0x7f6f28130f90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c009850 tx=0x7f6f1c0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947f7dafaa2d8718 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:14.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.453+0000 7f6f25ffb640 1 -- 172.21.15.102:0/77819144 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c033510 con 0x7f6f2809af20 2024-09-13T17:42:14.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.453+0000 7f6f25ffb640 1 -- 172.21.15.102:0/77819144 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f1c031050 con 0x7f6f2809af20 2024-09-13T17:42:14.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.453+0000 7f6f25ffb640 1 -- 172.21.15.102:0/77819144 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c0338e0 con 0x7f6f2809af20 2024-09-13T17:42:14.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.454+0000 7f739719d640 1 -- 172.21.15.102:0/1690485924 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f739012d8b0 msgr2=0x7f7390135530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.454+0000 7f739719d640 1 --2- 172.21.15.102:0/1690485924 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f739012d8b0 0x7f7390135530 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7388009ec0 tx=0x7f7388009780 comp rx=0 tx=0).stop 2024-09-13T17:42:14.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.454+0000 7f739719d640 1 -- 172.21.15.102:0/1690485924 shutdown_connections 2024-09-13T17:42:14.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.454+0000 7f739719d640 1 --2- 172.21.15.102:0/1690485924 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f739012d8b0 0x7f7390135530 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.454+0000 7f739719d640 1 --2- 172.21.15.102:0/1690485924 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f739006dca0 0x7f739006d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.454+0000 7f739719d640 1 -- 172.21.15.102:0/1690485924 >> 172.21.15.102:0/1690485924 conn(0x7f7390067bb0 msgr2=0x7f73900a08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:14.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.454+0000 7f739719d640 1 -- 172.21.15.102:0/1690485924 shutdown_connections 2024-09-13T17:42:14.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.454+0000 7f739719d640 1 -- 172.21.15.102:0/1690485924 wait complete. 2024-09-13T17:42:14.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.454+0000 7f6f2d79d640 1 -- 172.21.15.102:0/77819144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f2809af20 msgr2=0x7f6f28130f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.454+0000 7f6f2d79d640 1 --2- 172.21.15.102:0/77819144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f2809af20 0x7f6f28130f90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c009850 tx=0x7f6f1c0090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.454+0000 7f6f2d79d640 1 -- 172.21.15.102:0/77819144 shutdown_connections 2024-09-13T17:42:14.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.454+0000 7f6f2d79d640 1 --2- 172.21.15.102:0/77819144 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6f2809b320 0x7f6f281334e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.454+0000 7f6f2d79d640 1 --2- 172.21.15.102:0/77819144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f2809af20 0x7f6f28130f90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.454+0000 7f6f2d79d640 1 -- 172.21.15.102:0/77819144 >> 172.21.15.102:0/77819144 conn(0x7f6f2809f8b0 msgr2=0x7f6f2809b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:14.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.454+0000 7f6f2d79d640 1 -- 172.21.15.102:0/77819144 shutdown_connections 2024-09-13T17:42:14.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.454+0000 7f6f2d79d640 1 -- 172.21.15.102:0/77819144 wait complete. 2024-09-13T17:42:14.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.455+0000 7f739719d640 1 Processor -- start 2024-09-13T17:42:14.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.455+0000 7f739719d640 1 -- start start 2024-09-13T17:42:14.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.455+0000 7f6f2d79d640 1 Processor -- start 2024-09-13T17:42:14.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.455+0000 7f6f2d79d640 1 -- start start 2024-09-13T17:42:14.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.455+0000 7f739719d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f739006dca0 0x7f7390124c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.456+0000 7f6f2d79d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6f2809af20 0x7f6f2812b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.456+0000 7f739719d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f739009f7f0 0x7f7390120e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.456+0000 7f739719d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f739012d8b0 0x7f7390121350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.456+0000 7f739719d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7390123310 con 0x7f739006dca0 2024-09-13T17:42:14.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.456+0000 7f739719d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f739006c190 con 0x7f739012d8b0 2024-09-13T17:42:14.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.456+0000 7f739719d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73901241e0 con 0x7f739009f7f0 2024-09-13T17:42:14.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.456+0000 7f738ffff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f739009f7f0 0x7f7390120e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.456+0000 7f739573a640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f739012d8b0 0x7f7390121350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.456+0000 7f7394f39640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f739006dca0 0x7f7390124c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.456+0000 7f739573a640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f739012d8b0 0x7f7390121350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35408/0 (socket says 172.21.15.102:35408) 2024-09-13T17:42:14.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.456+0000 7f738ffff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f739009f7f0 0x7f7390120e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:46660/0 (socket says 172.21.15.102:46660) 2024-09-13T17:42:14.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.456+0000 7f739573a640 1 -- 172.21.15.102:0/3043059133 learned_addr learned my addr 172.21.15.102:0/3043059133 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:14.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.456+0000 7f6f26ffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6f2809af20 0x7f6f2812b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.456+0000 7f6f26ffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6f2809af20 0x7f6f2812b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:46658/0 (socket says 172.21.15.102:46658) 2024-09-13T17:42:14.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.456+0000 7f6f2d79d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6f2809b320 0x7f6f28127860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.456+0000 7f7394f39640 1 -- 172.21.15.102:0/3043059133 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f739009f7f0 msgr2=0x7f7390120e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.456+0000 7f7394f39640 1 --2- 172.21.15.102:0/3043059133 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f739009f7f0 0x7f7390120e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.456+0000 7f7394f39640 1 -- 172.21.15.102:0/3043059133 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f739012d8b0 msgr2=0x7f7390121350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.456+0000 7f6f2d79d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f2809c2a0 0x7f6f28128060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.456+0000 7f6f2d79d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2812af20 con 0x7f6f2809c2a0 2024-09-13T17:42:14.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.456+0000 7f6f2d79d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f28127da0 con 0x7f6f2809b320 2024-09-13T17:42:14.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.456+0000 7f6f2d79d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2812ac40 con 0x7f6f2809af20 2024-09-13T17:42:14.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.456+0000 7f7394f39640 1 --2- 172.21.15.102:0/3043059133 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f739012d8b0 0x7f7390121350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.456+0000 7f6f26ffd640 1 -- 172.21.15.102:0/3381457373 learned_addr learned my addr 172.21.15.102:0/3381457373 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:14.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.456+0000 7f7394f39640 1 -- 172.21.15.102:0/3043059133 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7388009a20 con 0x7f739006dca0 2024-09-13T17:42:14.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.457+0000 7f7394f39640 1 --2- 172.21.15.102:0/3043059133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f739006dca0 0x7f7390124c80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f738400ea80 tx=0x7f73840090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:14.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.456+0000 7f6f267fc640 1 --2- 172.21.15.102:0/3381457373 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6f2809b320 0x7f6f28127860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.456+0000 7f6f277fe640 1 --2- 172.21.15.102:0/3381457373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f2809c2a0 0x7f6f28128060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.457+0000 7f6f26ffd640 1 -- 172.21.15.102:0/3381457373 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6f2809b320 msgr2=0x7f6f28127860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.457+0000 7f6f26ffd640 1 --2- 172.21.15.102:0/3381457373 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6f2809b320 0x7f6f28127860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.457+0000 7f6f26ffd640 1 -- 172.21.15.102:0/3381457373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f2809c2a0 msgr2=0x7f6f28128060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.457+0000 7f6f26ffd640 1 --2- 172.21.15.102:0/3381457373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f2809c2a0 0x7f6f28128060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.457+0000 7f6f26ffd640 1 -- 172.21.15.102:0/3381457373 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f1c009390 con 0x7f6f2809af20 2024-09-13T17:42:14.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.457+0000 7f738ffff640 1 --2- 172.21.15.102:0/3043059133 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f739009f7f0 0x7f7390120e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:14.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.457+0000 7f739573a640 1 --2- 172.21.15.102:0/3043059133 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f739012d8b0 0x7f7390121350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:14.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.457+0000 7f6f26ffd640 1 --2- 172.21.15.102:0/3381457373 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6f2809af20 0x7f6f2812b420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6f1800c5b0 tx=0x7f6f18007db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:14.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.457+0000 7f738cff9640 1 -- 172.21.15.102:0/3043059133 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738400c800 con 0x7f739006dca0 2024-09-13T17:42:14.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.457+0000 7f738cff9640 1 -- 172.21.15.102:0/3043059133 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f738400c960 con 0x7f739006dca0 2024-09-13T17:42:14.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.457+0000 7f6f0effd640 1 -- 172.21.15.102:0/3381457373 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f18007360 con 0x7f6f2809af20 2024-09-13T17:42:14.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.457+0000 7f738cff9640 1 -- 172.21.15.102:0/3043059133 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738400cce0 con 0x7f739006dca0 2024-09-13T17:42:14.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.457+0000 7f739719d640 1 -- 172.21.15.102:0/3043059133 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7390069890 con 0x7f739006dca0 2024-09-13T17:42:14.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.457+0000 7f6f0effd640 1 -- 172.21.15.102:0/3381457373 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f180074e0 con 0x7f6f2809af20 2024-09-13T17:42:14.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.457+0000 7f6f2d79d640 1 -- 172.21.15.102:0/3381457373 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f28129450 con 0x7f6f2809af20 2024-09-13T17:42:14.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.457+0000 7f739719d640 1 -- 172.21.15.102:0/3043059133 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7390123e10 con 0x7f739006dca0 2024-09-13T17:42:14.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.457+0000 7f6f0effd640 1 -- 172.21.15.102:0/3381457373 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f180049f0 con 0x7f6f2809af20 2024-09-13T17:42:14.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.458+0000 7f6f2d79d640 1 -- 172.21.15.102:0/3381457373 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f28128740 con 0x7f6f2809af20 2024-09-13T17:42:14.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.458+0000 7f738cff9640 1 -- 172.21.15.102:0/3043059133 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f7384013680 con 0x7f739006dca0 2024-09-13T17:42:14.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.458+0000 7f6f0effd640 1 -- 172.21.15.102:0/3381457373 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f6f1801c3b0 con 0x7f6f2809af20 2024-09-13T17:42:14.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.459+0000 7f738cff9640 1 -- 172.21.15.102:0/3043059133 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7f7384046170 con 0x7f739006dca0 2024-09-13T17:42:14.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.459+0000 7f6f0effd640 1 -- 172.21.15.102:0/3381457373 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7f6f18046170 con 0x7f6f2809af20 2024-09-13T17:42:14.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.459+0000 7f739719d640 1 -- 172.21.15.102:0/3043059133 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f739006cc20 con 0x7f739006dca0 2024-09-13T17:42:14.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.459+0000 7f6f2d79d640 1 -- 172.21.15.102:0/3381457373 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f281295e0 con 0x7f6f2809af20 2024-09-13T17:42:14.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.463+0000 7f6f0effd640 1 -- 172.21.15.102:0/3381457373 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f6f18047060 con 0x7f6f2809af20 2024-09-13T17:42:14.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.468+0000 7f738cff9640 1 -- 172.21.15.102:0/3043059133 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f7384047060 con 0x7f739006dca0 2024-09-13T17:42:14.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.563+0000 7ffb2f0c0640 1 -- 172.21.15.102:0/4096352822 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ffb2809c7d0 con 0x7ffb2809de20 2024-09-13T17:42:14.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.563+0000 7ffb24ff9640 1 -- 172.21.15.102:0/4096352822 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ffb14045dd0 con 0x7ffb2809de20 2024-09-13T17:42:14.565 INFO:teuthology.orchestra.run.smithi102.stdout:38654705670 2024-09-13T17:42:14.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.566+0000 7ffb2f0c0640 1 -- 172.21.15.102:0/4096352822 >> v1:172.21.15.102:6816/34937 conn(0x7ffb0c031190 legacy=0x7ffb0c033640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.566+0000 7ffb2f0c0640 1 -- 172.21.15.102:0/4096352822 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffb2809de20 msgr2=0x7ffb28134750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.566+0000 7ffb2f0c0640 1 --2- 172.21.15.102:0/4096352822 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffb2809de20 0x7ffb28134750 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ffb28069730 tx=0x7ffb14032000 comp rx=0 tx=0).stop 2024-09-13T17:42:14.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.566+0000 7ffb2f0c0640 1 -- 172.21.15.102:0/4096352822 shutdown_connections 2024-09-13T17:42:14.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.566+0000 7ffb2f0c0640 1 --2- 172.21.15.102:0/4096352822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb28119ce0 0x7ffb28134c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.566+0000 7ffb2f0c0640 1 --2- 172.21.15.102:0/4096352822 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffb2809de20 0x7ffb28134750 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.566+0000 7ffb2f0c0640 1 --2- 172.21.15.102:0/4096352822 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffb28068920 0x7ffb28134210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.566+0000 7ffb2f0c0640 1 -- 172.21.15.102:0/4096352822 >> 172.21.15.102:0/4096352822 conn(0x7ffb2806f320 msgr2=0x7ffb280a0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:14.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.566+0000 7ffb2f0c0640 1 -- 172.21.15.102:0/4096352822 shutdown_connections 2024-09-13T17:42:14.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.566+0000 7ffb2f0c0640 1 -- 172.21.15.102:0/4096352822 wait complete. 2024-09-13T17:42:14.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.569+0000 7f32cd88b640 1 -- 172.21.15.102:0/3214981790 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f32c8098d40 con 0x7f32c809b350 2024-09-13T17:42:14.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.569+0000 7f32beffd640 1 -- 172.21.15.102:0/3214981790 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f32c0012e60 con 0x7f32c809b350 2024-09-13T17:42:14.571 INFO:teuthology.orchestra.run.smithi102.stdout:38654705672 2024-09-13T17:42:14.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.571+0000 7f32cd88b640 1 -- 172.21.15.102:0/3214981790 >> v1:172.21.15.102:6816/34937 conn(0x7f32a8031160 legacy=0x7f32a8033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.571+0000 7f32cd88b640 1 -- 172.21.15.102:0/3214981790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32c809b350 msgr2=0x7f32c8091000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.571+0000 7f32cd88b640 1 --2- 172.21.15.102:0/3214981790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32c809b350 0x7f32c8091000 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009e50 tx=0x7f32c000b040 comp rx=0 tx=0).stop 2024-09-13T17:42:14.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.571+0000 7f32cd88b640 1 -- 172.21.15.102:0/3214981790 shutdown_connections 2024-09-13T17:42:14.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.571+0000 7f32cd88b640 1 --2- 172.21.15.102:0/3214981790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32c809b350 0x7f32c8091000 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.571+0000 7f32cd88b640 1 --2- 172.21.15.102:0/3214981790 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32c809af20 0x7f32c8090ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.571+0000 7f32cd88b640 1 --2- 172.21.15.102:0/3214981790 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32c8097670 0x7f32c8094740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.571+0000 7f32cd88b640 1 -- 172.21.15.102:0/3214981790 >> 172.21.15.102:0/3214981790 conn(0x7f32c809f8b0 msgr2=0x7f32c8099800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:14.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.572+0000 7f32cd88b640 1 -- 172.21.15.102:0/3214981790 shutdown_connections 2024-09-13T17:42:14.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.572+0000 7f32cd88b640 1 -- 172.21.15.102:0/3214981790 wait complete. 2024-09-13T17:42:14.584 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705670 for osd.0 2024-09-13T17:42:14.584 DEBUG:teuthology.parallel:result is None 2024-09-13T17:42:14.585 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705672 got 38654705672 for osd.1 2024-09-13T17:42:14.585 DEBUG:teuthology.parallel:result is None 2024-09-13T17:42:14.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.625+0000 7f6f2d79d640 1 -- 172.21.15.102:0/3381457373 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6f28064600 con 0x7f6f2809af20 2024-09-13T17:42:14.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.625+0000 7f6f0effd640 1 -- 172.21.15.102:0/3381457373 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6f1801b360 con 0x7f6f2809af20 2024-09-13T17:42:14.627 INFO:teuthology.orchestra.run.smithi102.stdout:38654705669 2024-09-13T17:42:14.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.627+0000 7f6f2d79d640 1 -- 172.21.15.102:0/3381457373 >> v1:172.21.15.102:6816/34937 conn(0x7f6f14031160 legacy=0x7f6f14033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.627+0000 7f6f2d79d640 1 -- 172.21.15.102:0/3381457373 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6f2809af20 msgr2=0x7f6f2812b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.627+0000 7f6f2d79d640 1 --2- 172.21.15.102:0/3381457373 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6f2809af20 0x7f6f2812b420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6f1800c5b0 tx=0x7f6f18007db0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.627+0000 7f6f2d79d640 1 -- 172.21.15.102:0/3381457373 shutdown_connections 2024-09-13T17:42:14.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.627+0000 7f6f2d79d640 1 --2- 172.21.15.102:0/3381457373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f2809c2a0 0x7f6f28128060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.627+0000 7f6f2d79d640 1 --2- 172.21.15.102:0/3381457373 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6f2809b320 0x7f6f28127860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.627+0000 7f6f2d79d640 1 --2- 172.21.15.102:0/3381457373 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6f2809af20 0x7f6f2812b420 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.627+0000 7f6f2d79d640 1 -- 172.21.15.102:0/3381457373 >> 172.21.15.102:0/3381457373 conn(0x7f6f2809f8b0 msgr2=0x7f6f2809cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:14.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.627+0000 7f6f2d79d640 1 -- 172.21.15.102:0/3381457373 shutdown_connections 2024-09-13T17:42:14.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.627+0000 7f6f2d79d640 1 -- 172.21.15.102:0/3381457373 wait complete. 2024-09-13T17:42:14.639 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.3 2024-09-13T17:42:14.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.638+0000 7f739719d640 1 -- 172.21.15.102:0/3043059133 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7390064600 con 0x7f739006dca0 2024-09-13T17:42:14.640 DEBUG:teuthology.parallel:result is None 2024-09-13T17:42:14.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.638+0000 7f738cff9640 1 -- 172.21.15.102:0/3043059133 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7384013930 con 0x7f739006dca0 2024-09-13T17:42:14.640 INFO:teuthology.orchestra.run.smithi102.stdout:38654705669 2024-09-13T17:42:14.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.640+0000 7f739719d640 1 -- 172.21.15.102:0/3043059133 >> v1:172.21.15.102:6816/34937 conn(0x7f73740311b0 legacy=0x7f7374033660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.640+0000 7f739719d640 1 -- 172.21.15.102:0/3043059133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f739006dca0 msgr2=0x7f7390124c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.640+0000 7f739719d640 1 --2- 172.21.15.102:0/3043059133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f739006dca0 0x7f7390124c80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f738400ea80 tx=0x7f73840090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.640+0000 7f739719d640 1 -- 172.21.15.102:0/3043059133 shutdown_connections 2024-09-13T17:42:14.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.640+0000 7f739719d640 1 --2- 172.21.15.102:0/3043059133 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f739012d8b0 0x7f7390121350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.640+0000 7f739719d640 1 --2- 172.21.15.102:0/3043059133 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f739009f7f0 0x7f7390120e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.640+0000 7f739719d640 1 --2- 172.21.15.102:0/3043059133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f739006dca0 0x7f7390124c80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.640+0000 7f739719d640 1 -- 172.21.15.102:0/3043059133 >> 172.21.15.102:0/3043059133 conn(0x7f7390067bb0 msgr2=0x7f739012e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:14.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.640+0000 7f739719d640 1 -- 172.21.15.102:0/3043059133 shutdown_connections 2024-09-13T17:42:14.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.640+0000 7f739719d640 1 -- 172.21.15.102:0/3043059133 wait complete. 2024-09-13T17:42:14.652 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.2 2024-09-13T17:42:14.653 DEBUG:teuthology.parallel:result is None 2024-09-13T17:42:14.653 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T17:42:14.653 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:42:14.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.784+0000 7f4fd3d5d640 1 Processor -- start 2024-09-13T17:42:14.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.784+0000 7f4fd3d5d640 1 -- start start 2024-09-13T17:42:14.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.784+0000 7f4fd3d5d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4fcc12d590 0x7f4fcc12d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.784+0000 7f4fd3d5d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4fcc067840 0x7f4fcc1355d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.784+0000 7f4fd3d5d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fcc099ec0 con 0x7f4fcc12dea0 2024-09-13T17:42:14.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.784+0000 7f4fd3d5d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc12d390 con 0x7f4fcc12d590 2024-09-13T17:42:14.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.784+0000 7f4fd3d5d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc09bc30 con 0x7f4fcc067840 2024-09-13T17:42:14.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.784+0000 7f4fd1af9640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4fcc12d590 0x7f4fcc12d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.784+0000 7f4fd22fa640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4fcc067840 0x7f4fcc1355d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.784+0000 7f4fd12f8640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f4fcc12dea0 0x7f4fcc067130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:60880/0 (socket says 172.21.15.102:60880) 2024-09-13T17:42:14.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.784+0000 7f4fd12f8640 1 -- 172.21.15.102:0/476944011 learned_addr learned my addr 172.21.15.102:0/476944011 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:42:14.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.785+0000 7f4fd1af9640 1 -- 172.21.15.102:0/476944011 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4fcc067840 msgr2=0x7f4fcc1355d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:14.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.785+0000 7f4fd0af7640 1 -- 172.21.15.102:0/476944011 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3623622956 0 0) 0x7f4fcc099ec0 con 0x7f4fcc12dea0 2024-09-13T17:42:14.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.785+0000 7f4fd1af9640 1 --2- 172.21.15.102:0/476944011 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4fcc067840 0x7f4fcc1355d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.785+0000 7f4fd1af9640 1 -- 172.21.15.102:0/476944011 >> v1:172.21.15.102:6789/0 conn(0x7f4fcc12dea0 legacy=0x7f4fcc067130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.785+0000 7f4fd1af9640 1 -- 172.21.15.102:0/476944011 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fcc09aab0 con 0x7f4fcc12d590 2024-09-13T17:42:14.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.785+0000 7f4fd22fa640 1 --2- 172.21.15.102:0/476944011 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4fcc067840 0x7f4fcc1355d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:14.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.785+0000 7f4fd1af9640 1 --2- 172.21.15.102:0/476944011 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4fcc12d590 0x7f4fcc12d960 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009880 tx=0x7f4fc80090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2eceb1e74cf570a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:14.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.786+0000 7f4fd0af7640 1 -- 172.21.15.102:0/476944011 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc802acd0 con 0x7f4fcc12d590 2024-09-13T17:42:14.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.786+0000 7f4fd0af7640 1 -- 172.21.15.102:0/476944011 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc802ae30 con 0x7f4fcc12d590 2024-09-13T17:42:14.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.786+0000 7f4fd0af7640 1 -- 172.21.15.102:0/476944011 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8032670 con 0x7f4fcc12d590 2024-09-13T17:42:14.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.786+0000 7f4fd3d5d640 1 -- 172.21.15.102:0/476944011 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4fcc12d590 msgr2=0x7f4fcc12d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.786+0000 7f4fd3d5d640 1 --2- 172.21.15.102:0/476944011 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4fcc12d590 0x7f4fcc12d960 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009880 tx=0x7f4fc80090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.786+0000 7f4fd3d5d640 1 -- 172.21.15.102:0/476944011 shutdown_connections 2024-09-13T17:42:14.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.786+0000 7f4fd3d5d640 1 --2- 172.21.15.102:0/476944011 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4fcc067840 0x7f4fcc1355d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.786+0000 7f4fd3d5d640 1 --2- 172.21.15.102:0/476944011 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4fcc12d590 0x7f4fcc12d960 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.786+0000 7f4fd3d5d640 1 -- 172.21.15.102:0/476944011 >> 172.21.15.102:0/476944011 conn(0x7f4fcc05db50 msgr2=0x7f4fcc05df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:14.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.786+0000 7f4fd3d5d640 1 -- 172.21.15.102:0/476944011 shutdown_connections 2024-09-13T17:42:14.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.786+0000 7f4fd3d5d640 1 -- 172.21.15.102:0/476944011 wait complete. 2024-09-13T17:42:14.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.787+0000 7f4fd3d5d640 1 Processor -- start 2024-09-13T17:42:14.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.787+0000 7f4fd3d5d640 1 -- start start 2024-09-13T17:42:14.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.787+0000 7f4fd3d5d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4fcc067840 0x7f4fcc118c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.787+0000 7f4fd3d5d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4fcc12d590 0x7f4fcc1191b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.787+0000 7f4fd3d5d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4fcc12dea0 0x7f4fcc13c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.787+0000 7f4fd3d5d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc12cd60 con 0x7f4fcc12d590 2024-09-13T17:42:14.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.787+0000 7f4fd3d5d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc118a70 con 0x7f4fcc067840 2024-09-13T17:42:14.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.787+0000 7f4fd3d5d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc12d0a0 con 0x7f4fcc12dea0 2024-09-13T17:42:14.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.787+0000 7f4fd1af9640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4fcc067840 0x7f4fcc118c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.787+0000 7f4fd12f8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4fcc12d590 0x7f4fcc1191b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.787+0000 7f4fd22fa640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4fcc12dea0 0x7f4fcc13c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.787+0000 7f4fd1af9640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4fcc067840 0x7f4fcc118c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35430/0 (socket says 172.21.15.102:35430) 2024-09-13T17:42:14.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.787+0000 7f4fd1af9640 1 -- 172.21.15.102:0/3312380868 learned_addr learned my addr 172.21.15.102:0/3312380868 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:14.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.788+0000 7f4fd22fa640 1 -- 172.21.15.102:0/3312380868 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4fcc067840 msgr2=0x7f4fcc118c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.788+0000 7f4fd22fa640 1 --2- 172.21.15.102:0/3312380868 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4fcc067840 0x7f4fcc118c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.788+0000 7f4fd22fa640 1 -- 172.21.15.102:0/3312380868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4fcc12d590 msgr2=0x7f4fcc1191b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.788+0000 7f4fd22fa640 1 --2- 172.21.15.102:0/3312380868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4fcc12d590 0x7f4fcc1191b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.788+0000 7f4fd22fa640 1 -- 172.21.15.102:0/3312380868 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc4009a20 con 0x7f4fcc12dea0 2024-09-13T17:42:14.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.788+0000 7f4fd22fa640 1 --2- 172.21.15.102:0/3312380868 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4fcc12dea0 0x7f4fcc13c730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4fc400b460 tx=0x7f4fc400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:14.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.788+0000 7f4fc1ffb640 1 -- 172.21.15.102:0/3312380868 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4015070 con 0x7f4fcc12dea0 2024-09-13T17:42:14.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.788+0000 7f4fc1ffb640 1 -- 172.21.15.102:0/3312380868 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc4007550 con 0x7f4fcc12dea0 2024-09-13T17:42:14.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.788+0000 7f4fd3d5d640 1 -- 172.21.15.102:0/3312380868 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc8009390 con 0x7f4fcc12dea0 2024-09-13T17:42:14.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.788+0000 7f4fc1ffb640 1 -- 172.21.15.102:0/3312380868 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc40078f0 con 0x7f4fcc12dea0 2024-09-13T17:42:14.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.788+0000 7f4fd3d5d640 1 -- 172.21.15.102:0/3312380868 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fcc11c6d0 con 0x7f4fcc12dea0 2024-09-13T17:42:14.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.789+0000 7f4fc1ffb640 1 -- 172.21.15.102:0/3312380868 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f4fc4003020 con 0x7f4fcc12dea0 2024-09-13T17:42:14.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.789+0000 7f4fc1ffb640 1 -- 172.21.15.102:0/3312380868 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7f4fc40465d0 con 0x7f4fcc12dea0 2024-09-13T17:42:14.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.789+0000 7f4fd3d5d640 1 -- 172.21.15.102:0/3312380868 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fcc064000 con 0x7f4fcc12dea0 2024-09-13T17:42:14.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.793+0000 7f4fc1ffb640 1 -- 172.21.15.102:0/3312380868 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f4fc401d030 con 0x7f4fcc12dea0 2024-09-13T17:42:14.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.943+0000 7f4fd3d5d640 1 -- 172.21.15.102:0/3312380868 --> v1:172.21.15.102:6816/34937 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4fcc068800 con 0x7f4fb0031160 2024-09-13T17:42:14.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.948+0000 7f4fc1ffb640 1 -- 172.21.15.102:0/3312380868 <== mgr.4101 v1:172.21.15.102:6816/34937 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+271380 (unknown 2965378022 0 909147157) 0x7f4fcc068800 con 0x7f4fb0031160 2024-09-13T17:42:14.955 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:42:14.957 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-09-13T17:42:14.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.951+0000 7f4fd3d5d640 1 -- 172.21.15.102:0/3312380868 >> v1:172.21.15.102:6816/34937 conn(0x7f4fb0031160 legacy=0x7f4fb0033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.951+0000 7f4fd3d5d640 1 -- 172.21.15.102:0/3312380868 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4fcc12dea0 msgr2=0x7f4fcc13c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.951+0000 7f4fd3d5d640 1 --2- 172.21.15.102:0/3312380868 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4fcc12dea0 0x7f4fcc13c730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4fc400b460 tx=0x7f4fc400b040 comp rx=0 tx=0).stop 2024-09-13T17:42:14.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.952+0000 7f4fd3d5d640 1 -- 172.21.15.102:0/3312380868 shutdown_connections 2024-09-13T17:42:14.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.952+0000 7f4fd3d5d640 1 --2- 172.21.15.102:0/3312380868 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4fcc12dea0 0x7f4fcc13c730 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.952+0000 7f4fd3d5d640 1 --2- 172.21.15.102:0/3312380868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4fcc12d590 0x7f4fcc1191b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.952+0000 7f4fd3d5d640 1 --2- 172.21.15.102:0/3312380868 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4fcc067840 0x7f4fcc118c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.952+0000 7f4fd3d5d640 1 -- 172.21.15.102:0/3312380868 >> 172.21.15.102:0/3312380868 conn(0x7f4fcc05db50 msgr2=0x7f4fcc09e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:14.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.952+0000 7f4fd3d5d640 1 -- 172.21.15.102:0/3312380868 shutdown_connections 2024-09-13T17:42:14.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:14.952+0000 7f4fd3d5d640 1 -- 172.21.15.102:0/3312380868 wait complete. 2024-09-13T17:42:14.964 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-13T17:42:14.413816+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592725,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":125,"num_write_kb":1121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":93,"ondisk_log_size":93,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1191344,"kb_used_data":1904,"kb_used_omap":6,"kb_used_meta":1189369,"kb_avail":376296016,"statfs":{"total":386547056640,"available":385327120384,"internally_reserved":0,"allocated":1949696,"data_stored":1408590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6816,"internal_metadata":1217914208},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001349"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.334754+0000","last_change":"2024-09-13T17:42:07.332154+0000","last_active":"2024-09-13T17:42:07.334754+0000","last_peered":"2024-09-13T17:42:07.334754+0000","last_clean":"2024-09-13T17:42:07.334754+0000","last_became_active":"2024-09-13T17:42:07.331874+0000","last_became_peered":"2024-09-13T17:42:07.331874+0000","last_unstale":"2024-09-13T17:42:07.334754+0000","last_undegraded":"2024-09-13T17:42:07.334754+0000","last_fullsized":"2024-09-13T17:42:07.334754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:10:22.943375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.329161+0000","last_change":"2024-09-13T17:42:07.326086+0000","last_active":"2024-09-13T17:42:07.329161+0000","last_peered":"2024-09-13T17:42:07.329161+0000","last_clean":"2024-09-13T17:42:07.329161+0000","last_became_active":"2024-09-13T17:42:07.325830+0000","last_became_peered":"2024-09-13T17:42:07.325830+0000","last_unstale":"2024-09-13T17:42:07.329161+0000","last_undegraded":"2024-09-13T17:42:07.329161+0000","last_fullsized":"2024-09-13T17:42:07.329161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:54.719512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338441+0000","last_change":"2024-09-13T17:42:07.326953+0000","last_active":"2024-09-13T17:42:07.338441+0000","last_peered":"2024-09-13T17:42:07.338441+0000","last_clean":"2024-09-13T17:42:07.338441+0000","last_became_active":"2024-09-13T17:42:07.326717+0000","last_became_peered":"2024-09-13T17:42:07.326717+0000","last_unstale":"2024-09-13T17:42:07.338441+0000","last_undegraded":"2024-09-13T17:42:07.338441+0000","last_fullsized":"2024-09-13T17:42:07.338441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:20.647404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338424+0000","last_change":"2024-09-13T17:42:07.326356+0000","last_active":"2024-09-13T17:42:07.338424+0000","last_peered":"2024-09-13T17:42:07.338424+0000","last_clean":"2024-09-13T17:42:07.338424+0000","last_became_active":"2024-09-13T17:42:07.326127+0000","last_became_peered":"2024-09-13T17:42:07.326127+0000","last_unstale":"2024-09-13T17:42:07.338424+0000","last_undegraded":"2024-09-13T17:42:07.338424+0000","last_fullsized":"2024-09-13T17:42:07.338424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:44.405426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338304+0000","last_change":"2024-09-13T17:42:07.330092+0000","last_active":"2024-09-13T17:42:07.338304+0000","last_peered":"2024-09-13T17:42:07.338304+0000","last_clean":"2024-09-13T17:42:07.338304+0000","last_became_active":"2024-09-13T17:42:07.327658+0000","last_became_peered":"2024-09-13T17:42:07.327658+0000","last_unstale":"2024-09-13T17:42:07.338304+0000","last_undegraded":"2024-09-13T17:42:07.338304+0000","last_fullsized":"2024-09-13T17:42:07.338304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:24:02.920430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.324569+0000","last_change":"2024-09-13T17:42:07.303609+0000","last_active":"2024-09-13T17:42:07.324569+0000","last_peered":"2024-09-13T17:42:07.324569+0000","last_clean":"2024-09-13T17:42:07.324569+0000","last_became_active":"2024-09-13T17:42:07.303384+0000","last_became_peered":"2024-09-13T17:42:07.303384+0000","last_unstale":"2024-09-13T17:42:07.324569+0000","last_undegraded":"2024-09-13T17:42:07.324569+0000","last_fullsized":"2024-09-13T17:42:07.324569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:43:56.000592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.326264+0000","last_change":"2024-09-13T17:42:07.307630+0000","last_active":"2024-09-13T17:42:07.326264+0000","last_peered":"2024-09-13T17:42:07.326264+0000","last_clean":"2024-09-13T17:42:07.326264+0000","last_became_active":"2024-09-13T17:42:07.307382+0000","last_became_peered":"2024-09-13T17:42:07.307382+0000","last_unstale":"2024-09-13T17:42:07.326264+0000","last_undegraded":"2024-09-13T17:42:07.326264+0000","last_fullsized":"2024-09-13T17:42:07.326264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:26:32.846287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.326676+0000","last_change":"2024-09-13T17:42:07.320736+0000","last_active":"2024-09-13T17:42:07.326676+0000","last_peered":"2024-09-13T17:42:07.326676+0000","last_clean":"2024-09-13T17:42:07.326676+0000","last_became_active":"2024-09-13T17:42:07.319801+0000","last_became_peered":"2024-09-13T17:42:07.319801+0000","last_unstale":"2024-09-13T17:42:07.326676+0000","last_undegraded":"2024-09-13T17:42:07.326676+0000","last_fullsized":"2024-09-13T17:42:07.326676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:28:32.030397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.339799+0000","last_change":"2024-09-13T17:42:07.335208+0000","last_active":"2024-09-13T17:42:07.339799+0000","last_peered":"2024-09-13T17:42:07.339799+0000","last_clean":"2024-09-13T17:42:07.339799+0000","last_became_active":"2024-09-13T17:42:07.334956+0000","last_became_peered":"2024-09-13T17:42:07.334956+0000","last_unstale":"2024-09-13T17:42:07.339799+0000","last_undegraded":"2024-09-13T17:42:07.339799+0000","last_fullsized":"2024-09-13T17:42:07.339799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:59:53.544086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.325383+0000","last_change":"2024-09-13T17:42:07.317057+0000","last_active":"2024-09-13T17:42:07.325383+0000","last_peered":"2024-09-13T17:42:07.325383+0000","last_clean":"2024-09-13T17:42:07.325383+0000","last_became_active":"2024-09-13T17:42:07.316429+0000","last_became_peered":"2024-09-13T17:42:07.316429+0000","last_unstale":"2024-09-13T17:42:07.325383+0000","last_undegraded":"2024-09-13T17:42:07.325383+0000","last_fullsized":"2024-09-13T17:42:07.325383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:59:53.544086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.325590+0000","last_change":"2024-09-13T17:42:07.321106+0000","last_active":"2024-09-13T17:42:07.325590+0000","last_peered":"2024-09-13T17:42:07.325590+0000","last_clean":"2024-09-13T17:42:07.325590+0000","last_became_active":"2024-09-13T17:42:07.320827+0000","last_became_peered":"2024-09-13T17:42:07.320827+0000","last_unstale":"2024-09-13T17:42:07.325590+0000","last_undegraded":"2024-09-13T17:42:07.325590+0000","last_fullsized":"2024-09-13T17:42:07.325590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:34:34.794230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.334785+0000","last_change":"2024-09-13T17:42:07.331865+0000","last_active":"2024-09-13T17:42:07.334785+0000","last_peered":"2024-09-13T17:42:07.334785+0000","last_clean":"2024-09-13T17:42:07.334785+0000","last_became_active":"2024-09-13T17:42:07.331419+0000","last_became_peered":"2024-09-13T17:42:07.331419+0000","last_unstale":"2024-09-13T17:42:07.334785+0000","last_undegraded":"2024-09-13T17:42:07.334785+0000","last_fullsized":"2024-09-13T17:42:07.334785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:48.155823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.335811+0000","last_change":"2024-09-13T17:42:07.333296+0000","last_active":"2024-09-13T17:42:07.335811+0000","last_peered":"2024-09-13T17:42:07.335811+0000","last_clean":"2024-09-13T17:42:07.335811+0000","last_became_active":"2024-09-13T17:42:07.332898+0000","last_became_peered":"2024-09-13T17:42:07.332898+0000","last_unstale":"2024-09-13T17:42:07.335811+0000","last_undegraded":"2024-09-13T17:42:07.335811+0000","last_fullsized":"2024-09-13T17:42:07.335811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:20.241542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338928+0000","last_change":"2024-09-13T17:42:07.324771+0000","last_active":"2024-09-13T17:42:07.338928+0000","last_peered":"2024-09-13T17:42:07.338928+0000","last_clean":"2024-09-13T17:42:07.338928+0000","last_became_active":"2024-09-13T17:42:07.323820+0000","last_became_peered":"2024-09-13T17:42:07.323820+0000","last_unstale":"2024-09-13T17:42:07.338928+0000","last_undegraded":"2024-09-13T17:42:07.338928+0000","last_fullsized":"2024-09-13T17:42:07.338928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:24:02.920430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.337730+0000","last_change":"2024-09-13T17:42:07.325827+0000","last_active":"2024-09-13T17:42:07.337730+0000","last_peered":"2024-09-13T17:42:07.337730+0000","last_clean":"2024-09-13T17:42:07.337730+0000","last_became_active":"2024-09-13T17:42:07.325589+0000","last_became_peered":"2024-09-13T17:42:07.325589+0000","last_unstale":"2024-09-13T17:42:07.337730+0000","last_undegraded":"2024-09-13T17:42:07.337730+0000","last_fullsized":"2024-09-13T17:42:07.337730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:43.466051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.334093+0000","last_change":"2024-09-13T17:42:07.315478+0000","last_active":"2024-09-13T17:42:07.334093+0000","last_peered":"2024-09-13T17:42:07.334093+0000","last_clean":"2024-09-13T17:42:07.334093+0000","last_became_active":"2024-09-13T17:42:07.315218+0000","last_became_peered":"2024-09-13T17:42:07.315218+0000","last_unstale":"2024-09-13T17:42:07.334093+0000","last_undegraded":"2024-09-13T17:42:07.334093+0000","last_fullsized":"2024-09-13T17:42:07.334093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:06:44.563276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338279+0000","last_change":"2024-09-13T17:42:07.328192+0000","last_active":"2024-09-13T17:42:07.338279+0000","last_peered":"2024-09-13T17:42:07.338279+0000","last_clean":"2024-09-13T17:42:07.338279+0000","last_became_active":"2024-09-13T17:42:07.327951+0000","last_became_peered":"2024-09-13T17:42:07.327951+0000","last_unstale":"2024-09-13T17:42:07.338279+0000","last_undegraded":"2024-09-13T17:42:07.338279+0000","last_fullsized":"2024-09-13T17:42:07.338279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:28:32.030397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.320769+0000","last_change":"2024-09-13T17:42:07.314816+0000","last_active":"2024-09-13T17:42:07.320769+0000","last_peered":"2024-09-13T17:42:07.320769+0000","last_clean":"2024-09-13T17:42:07.320769+0000","last_became_active":"2024-09-13T17:42:07.314464+0000","last_became_peered":"2024-09-13T17:42:07.314464+0000","last_unstale":"2024-09-13T17:42:07.320769+0000","last_undegraded":"2024-09-13T17:42:07.320769+0000","last_fullsized":"2024-09-13T17:42:07.320769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:17.483066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.335387+0000","last_change":"2024-09-13T17:42:07.332269+0000","last_active":"2024-09-13T17:42:07.335387+0000","last_peered":"2024-09-13T17:42:07.335387+0000","last_clean":"2024-09-13T17:42:07.335387+0000","last_became_active":"2024-09-13T17:42:07.332040+0000","last_became_peered":"2024-09-13T17:42:07.332040+0000","last_unstale":"2024-09-13T17:42:07.335387+0000","last_undegraded":"2024-09-13T17:42:07.335387+0000","last_fullsized":"2024-09-13T17:42:07.335387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:48:49.368747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.340627+0000","last_change":"2024-09-13T17:42:07.335425+0000","last_active":"2024-09-13T17:42:07.340627+0000","last_peered":"2024-09-13T17:42:07.340627+0000","last_clean":"2024-09-13T17:42:07.340627+0000","last_became_active":"2024-09-13T17:42:07.335173+0000","last_became_peered":"2024-09-13T17:42:07.335173+0000","last_unstale":"2024-09-13T17:42:07.340627+0000","last_undegraded":"2024-09-13T17:42:07.340627+0000","last_fullsized":"2024-09-13T17:42:07.340627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:00:10.355609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.340514+0000","last_change":"2024-09-13T17:42:07.331183+0000","last_active":"2024-09-13T17:42:07.340514+0000","last_peered":"2024-09-13T17:42:07.340514+0000","last_clean":"2024-09-13T17:42:07.340514+0000","last_became_active":"2024-09-13T17:42:07.330936+0000","last_became_peered":"2024-09-13T17:42:07.330936+0000","last_unstale":"2024-09-13T17:42:07.340514+0000","last_undegraded":"2024-09-13T17:42:07.340514+0000","last_fullsized":"2024-09-13T17:42:07.340514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:42:38.505262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.337694+0000","last_change":"2024-09-13T17:42:07.325877+0000","last_active":"2024-09-13T17:42:07.337694+0000","last_peered":"2024-09-13T17:42:07.337694+0000","last_clean":"2024-09-13T17:42:07.337694+0000","last_became_active":"2024-09-13T17:42:07.325657+0000","last_became_peered":"2024-09-13T17:42:07.325657+0000","last_unstale":"2024-09-13T17:42:07.337694+0000","last_undegraded":"2024-09-13T17:42:07.337694+0000","last_fullsized":"2024-09-13T17:42:07.337694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:48.663311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331637+0000","last_change":"2024-09-13T17:42:07.322059+0000","last_active":"2024-09-13T17:42:07.331637+0000","last_peered":"2024-09-13T17:42:07.331637+0000","last_clean":"2024-09-13T17:42:07.331637+0000","last_became_active":"2024-09-13T17:42:07.312536+0000","last_became_peered":"2024-09-13T17:42:07.312536+0000","last_unstale":"2024-09-13T17:42:07.331637+0000","last_undegraded":"2024-09-13T17:42:07.331637+0000","last_fullsized":"2024-09-13T17:42:07.331637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:14:08.780968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.340774+0000","last_change":"2024-09-13T17:42:07.336557+0000","last_active":"2024-09-13T17:42:07.340774+0000","last_peered":"2024-09-13T17:42:07.340774+0000","last_clean":"2024-09-13T17:42:07.340774+0000","last_became_active":"2024-09-13T17:42:07.335820+0000","last_became_peered":"2024-09-13T17:42:07.335820+0000","last_unstale":"2024-09-13T17:42:07.340774+0000","last_undegraded":"2024-09-13T17:42:07.340774+0000","last_fullsized":"2024-09-13T17:42:07.340774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:20.647404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.342633+0000","last_change":"2024-09-13T17:42:07.337695+0000","last_active":"2024-09-13T17:42:07.342633+0000","last_peered":"2024-09-13T17:42:07.342633+0000","last_clean":"2024-09-13T17:42:07.342633+0000","last_became_active":"2024-09-13T17:42:07.337419+0000","last_became_peered":"2024-09-13T17:42:07.337419+0000","last_unstale":"2024-09-13T17:42:07.342633+0000","last_undegraded":"2024-09-13T17:42:07.342633+0000","last_fullsized":"2024-09-13T17:42:07.342633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:22:40.880345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.325365+0000","last_change":"2024-09-13T17:42:07.315008+0000","last_active":"2024-09-13T17:42:07.325365+0000","last_peered":"2024-09-13T17:42:07.325365+0000","last_clean":"2024-09-13T17:42:07.325365+0000","last_became_active":"2024-09-13T17:42:07.313477+0000","last_became_peered":"2024-09-13T17:42:07.313477+0000","last_unstale":"2024-09-13T17:42:07.325365+0000","last_undegraded":"2024-09-13T17:42:07.325365+0000","last_fullsized":"2024-09-13T17:42:07.325365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:25.632002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331694+0000","last_change":"2024-09-13T17:42:07.317949+0000","last_active":"2024-09-13T17:42:07.331694+0000","last_peered":"2024-09-13T17:42:07.331694+0000","last_clean":"2024-09-13T17:42:07.331694+0000","last_became_active":"2024-09-13T17:42:07.317720+0000","last_became_peered":"2024-09-13T17:42:07.317720+0000","last_unstale":"2024-09-13T17:42:07.331694+0000","last_undegraded":"2024-09-13T17:42:07.331694+0000","last_fullsized":"2024-09-13T17:42:07.331694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:31.378417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.324990+0000","last_change":"2024-09-13T17:42:07.313528+0000","last_active":"2024-09-13T17:42:07.324990+0000","last_peered":"2024-09-13T17:42:07.324990+0000","last_clean":"2024-09-13T17:42:07.324990+0000","last_became_active":"2024-09-13T17:42:07.313092+0000","last_became_peered":"2024-09-13T17:42:07.313092+0000","last_unstale":"2024-09-13T17:42:07.324990+0000","last_undegraded":"2024-09-13T17:42:07.324990+0000","last_fullsized":"2024-09-13T17:42:07.324990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:48:49.368747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331153+0000","last_change":"2024-09-13T17:42:07.314816+0000","last_active":"2024-09-13T17:42:07.331153+0000","last_peered":"2024-09-13T17:42:07.331153+0000","last_clean":"2024-09-13T17:42:07.331153+0000","last_became_active":"2024-09-13T17:42:07.314574+0000","last_became_peered":"2024-09-13T17:42:07.314574+0000","last_unstale":"2024-09-13T17:42:07.331153+0000","last_undegraded":"2024-09-13T17:42:07.331153+0000","last_fullsized":"2024-09-13T17:42:07.331153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:03:35.976183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338349+0000","last_change":"2024-09-13T17:42:07.328522+0000","last_active":"2024-09-13T17:42:07.338349+0000","last_peered":"2024-09-13T17:42:07.338349+0000","last_clean":"2024-09-13T17:42:07.338349+0000","last_became_active":"2024-09-13T17:42:07.327893+0000","last_became_peered":"2024-09-13T17:42:07.327893+0000","last_unstale":"2024-09-13T17:42:07.338349+0000","last_undegraded":"2024-09-13T17:42:07.338349+0000","last_fullsized":"2024-09-13T17:42:07.338349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:02:50.060149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331620+0000","last_change":"2024-09-13T17:42:07.321900+0000","last_active":"2024-09-13T17:42:07.331620+0000","last_peered":"2024-09-13T17:42:07.331620+0000","last_clean":"2024-09-13T17:42:07.331620+0000","last_became_active":"2024-09-13T17:42:07.319230+0000","last_became_peered":"2024-09-13T17:42:07.319230+0000","last_unstale":"2024-09-13T17:42:07.331620+0000","last_undegraded":"2024-09-13T17:42:07.331620+0000","last_fullsized":"2024-09-13T17:42:07.331620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:50:46.442172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.326655+0000","last_change":"2024-09-13T17:42:07.315846+0000","last_active":"2024-09-13T17:42:07.326655+0000","last_peered":"2024-09-13T17:42:07.326655+0000","last_clean":"2024-09-13T17:42:07.326655+0000","last_became_active":"2024-09-13T17:42:07.315513+0000","last_became_peered":"2024-09-13T17:42:07.315513+0000","last_unstale":"2024-09-13T17:42:07.326655+0000","last_undegraded":"2024-09-13T17:42:07.326655+0000","last_fullsized":"2024-09-13T17:42:07.326655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:56:21.343829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331104+0000","last_change":"2024-09-13T17:42:07.315400+0000","last_active":"2024-09-13T17:42:07.331104+0000","last_peered":"2024-09-13T17:42:07.331104+0000","last_clean":"2024-09-13T17:42:07.331104+0000","last_became_active":"2024-09-13T17:42:07.315131+0000","last_became_peered":"2024-09-13T17:42:07.315131+0000","last_unstale":"2024-09-13T17:42:07.331104+0000","last_undegraded":"2024-09-13T17:42:07.331104+0000","last_fullsized":"2024-09-13T17:42:07.331104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:21:39.335847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338775+0000","last_change":"2024-09-13T17:42:06.308504+0000","last_active":"2024-09-13T17:42:07.338775+0000","last_peered":"2024-09-13T17:42:07.338775+0000","last_clean":"2024-09-13T17:42:07.338775+0000","last_became_active":"2024-09-13T17:42:06.308199+0000","last_became_peered":"2024-09-13T17:42:06.308199+0000","last_unstale":"2024-09-13T17:42:07.338775+0000","last_undegraded":"2024-09-13T17:42:07.338775+0000","last_fullsized":"2024-09-13T17:42:07.338775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:05.930342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331918+0000","last_change":"2024-09-13T17:42:07.322597+0000","last_active":"2024-09-13T17:42:07.331918+0000","last_peered":"2024-09-13T17:42:07.331918+0000","last_clean":"2024-09-13T17:42:07.331918+0000","last_became_active":"2024-09-13T17:42:07.321945+0000","last_became_peered":"2024-09-13T17:42:07.321945+0000","last_unstale":"2024-09-13T17:42:07.331918+0000","last_undegraded":"2024-09-13T17:42:07.331918+0000","last_fullsized":"2024-09-13T17:42:07.331918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:03:28.132876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.301168+0000","last_change":"2024-09-13T17:42:06.306367+0000","last_active":"2024-09-13T17:42:07.301168+0000","last_peered":"2024-09-13T17:42:07.301168+0000","last_clean":"2024-09-13T17:42:07.301168+0000","last_became_active":"2024-09-13T17:42:06.306146+0000","last_became_peered":"2024-09-13T17:42:06.306146+0000","last_unstale":"2024-09-13T17:42:07.301168+0000","last_undegraded":"2024-09-13T17:42:07.301168+0000","last_fullsized":"2024-09-13T17:42:07.301168+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:45:01.750995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338414+0000","last_change":"2024-09-13T17:42:07.329528+0000","last_active":"2024-09-13T17:42:07.338414+0000","last_peered":"2024-09-13T17:42:07.338414+0000","last_clean":"2024-09-13T17:42:07.338414+0000","last_became_active":"2024-09-13T17:42:07.329296+0000","last_became_peered":"2024-09-13T17:42:07.329296+0000","last_unstale":"2024-09-13T17:42:07.338414+0000","last_undegraded":"2024-09-13T17:42:07.338414+0000","last_fullsized":"2024-09-13T17:42:07.338414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:33:57.543704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.294808+0000","last_change":"2024-09-13T17:42:06.312798+0000","last_active":"2024-09-13T17:42:07.294808+0000","last_peered":"2024-09-13T17:42:07.294808+0000","last_clean":"2024-09-13T17:42:07.294808+0000","last_became_active":"2024-09-13T17:42:06.312388+0000","last_became_peered":"2024-09-13T17:42:06.312388+0000","last_unstale":"2024-09-13T17:42:07.294808+0000","last_undegraded":"2024-09-13T17:42:07.294808+0000","last_fullsized":"2024-09-13T17:42:07.294808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:56:20.339850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331165+0000","last_change":"2024-09-13T17:42:07.315290+0000","last_active":"2024-09-13T17:42:07.331165+0000","last_peered":"2024-09-13T17:42:07.331165+0000","last_clean":"2024-09-13T17:42:07.331165+0000","last_became_active":"2024-09-13T17:42:07.314670+0000","last_became_peered":"2024-09-13T17:42:07.314670+0000","last_unstale":"2024-09-13T17:42:07.331165+0000","last_undegraded":"2024-09-13T17:42:07.331165+0000","last_fullsized":"2024-09-13T17:42:07.331165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:00:24.969812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.340570+0000","last_change":"2024-09-13T17:42:06.305806+0000","last_active":"2024-09-13T17:42:07.340570+0000","last_peered":"2024-09-13T17:42:07.340570+0000","last_clean":"2024-09-13T17:42:07.340570+0000","last_became_active":"2024-09-13T17:42:06.305576+0000","last_became_peered":"2024-09-13T17:42:06.305576+0000","last_unstale":"2024-09-13T17:42:07.340570+0000","last_undegraded":"2024-09-13T17:42:07.340570+0000","last_fullsized":"2024-09-13T17:42:07.340570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:47:16.711770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.337092+0000","last_change":"2024-09-13T17:42:06.304106+0000","last_active":"2024-09-13T17:42:07.337092+0000","last_peered":"2024-09-13T17:42:07.337092+0000","last_clean":"2024-09-13T17:42:07.337092+0000","last_became_active":"2024-09-13T17:42:06.303845+0000","last_became_peered":"2024-09-13T17:42:06.303845+0000","last_unstale":"2024-09-13T17:42:07.337092+0000","last_undegraded":"2024-09-13T17:42:07.337092+0000","last_fullsized":"2024-09-13T17:42:07.337092+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:36:02.512243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.334525+0000","last_change":"2024-09-13T17:42:07.331147+0000","last_active":"2024-09-13T17:42:07.334525+0000","last_peered":"2024-09-13T17:42:07.334525+0000","last_clean":"2024-09-13T17:42:07.334525+0000","last_became_active":"2024-09-13T17:42:07.329386+0000","last_became_peered":"2024-09-13T17:42:07.329386+0000","last_unstale":"2024-09-13T17:42:07.334525+0000","last_undegraded":"2024-09-13T17:42:07.334525+0000","last_fullsized":"2024-09-13T17:42:07.334525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:59:21.210970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.321593+0000","last_change":"2024-09-13T17:42:06.304872+0000","last_active":"2024-09-13T17:42:07.321593+0000","last_peered":"2024-09-13T17:42:07.321593+0000","last_clean":"2024-09-13T17:42:07.321593+0000","last_became_active":"2024-09-13T17:42:06.304639+0000","last_became_peered":"2024-09-13T17:42:06.304639+0000","last_unstale":"2024-09-13T17:42:07.321593+0000","last_undegraded":"2024-09-13T17:42:07.321593+0000","last_fullsized":"2024-09-13T17:42:07.321593+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:40.147410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.340505+0000","last_change":"2024-09-13T17:42:07.331235+0000","last_active":"2024-09-13T17:42:07.340505+0000","last_peered":"2024-09-13T17:42:07.340505+0000","last_clean":"2024-09-13T17:42:07.340505+0000","last_became_active":"2024-09-13T17:42:07.330822+0000","last_became_peered":"2024-09-13T17:42:07.330822+0000","last_unstale":"2024-09-13T17:42:07.340505+0000","last_undegraded":"2024-09-13T17:42:07.340505+0000","last_fullsized":"2024-09-13T17:42:07.340505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:38:04.013092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.340781+0000","last_change":"2024-09-13T17:42:06.310319+0000","last_active":"2024-09-13T17:42:07.340781+0000","last_peered":"2024-09-13T17:42:07.340781+0000","last_clean":"2024-09-13T17:42:07.340781+0000","last_became_active":"2024-09-13T17:42:06.310085+0000","last_became_peered":"2024-09-13T17:42:06.310085+0000","last_unstale":"2024-09-13T17:42:07.340781+0000","last_undegraded":"2024-09-13T17:42:07.340781+0000","last_fullsized":"2024-09-13T17:42:07.340781+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:53.715533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.335838+0000","last_change":"2024-09-13T17:42:07.333488+0000","last_active":"2024-09-13T17:42:07.335838+0000","last_peered":"2024-09-13T17:42:07.335838+0000","last_clean":"2024-09-13T17:42:07.335838+0000","last_became_active":"2024-09-13T17:42:07.332200+0000","last_became_peered":"2024-09-13T17:42:07.332200+0000","last_unstale":"2024-09-13T17:42:07.335838+0000","last_undegraded":"2024-09-13T17:42:07.335838+0000","last_fullsized":"2024-09-13T17:42:07.335838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:52.666716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.303967+0000","last_change":"2024-09-13T17:42:06.308854+0000","last_active":"2024-09-13T17:42:07.303967+0000","last_peered":"2024-09-13T17:42:07.303967+0000","last_clean":"2024-09-13T17:42:07.303967+0000","last_became_active":"2024-09-13T17:42:06.307173+0000","last_became_peered":"2024-09-13T17:42:06.307173+0000","last_unstale":"2024-09-13T17:42:07.303967+0000","last_undegraded":"2024-09-13T17:42:07.303967+0000","last_fullsized":"2024-09-13T17:42:07.303967+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:59.621392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338229+0000","last_change":"2024-09-13T17:42:07.327431+0000","last_active":"2024-09-13T17:42:07.338229+0000","last_peered":"2024-09-13T17:42:07.338229+0000","last_clean":"2024-09-13T17:42:07.338229+0000","last_became_active":"2024-09-13T17:42:07.327202+0000","last_became_peered":"2024-09-13T17:42:07.327202+0000","last_unstale":"2024-09-13T17:42:07.338229+0000","last_undegraded":"2024-09-13T17:42:07.338229+0000","last_fullsized":"2024-09-13T17:42:07.338229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:42:38.505262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338801+0000","last_change":"2024-09-13T17:42:06.308795+0000","last_active":"2024-09-13T17:42:07.338801+0000","last_peered":"2024-09-13T17:42:07.338801+0000","last_clean":"2024-09-13T17:42:07.338801+0000","last_became_active":"2024-09-13T17:42:06.308557+0000","last_became_peered":"2024-09-13T17:42:06.308557+0000","last_unstale":"2024-09-13T17:42:07.338801+0000","last_undegraded":"2024-09-13T17:42:07.338801+0000","last_fullsized":"2024-09-13T17:42:07.338801+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:51.662738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338442+0000","last_change":"2024-09-13T17:42:07.327849+0000","last_active":"2024-09-13T17:42:07.338442+0000","last_peered":"2024-09-13T17:42:07.338442+0000","last_clean":"2024-09-13T17:42:07.338442+0000","last_became_active":"2024-09-13T17:42:07.327578+0000","last_became_peered":"2024-09-13T17:42:07.327578+0000","last_unstale":"2024-09-13T17:42:07.338442+0000","last_undegraded":"2024-09-13T17:42:07.338442+0000","last_fullsized":"2024-09-13T17:42:07.338442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:34:34.794230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.294395+0000","last_change":"2024-09-13T17:42:06.310708+0000","last_active":"2024-09-13T17:42:07.294395+0000","last_peered":"2024-09-13T17:42:07.294395+0000","last_clean":"2024-09-13T17:42:07.294395+0000","last_became_active":"2024-09-13T17:42:06.310476+0000","last_became_peered":"2024-09-13T17:42:06.310476+0000","last_unstale":"2024-09-13T17:42:07.294395+0000","last_undegraded":"2024-09-13T17:42:07.294395+0000","last_fullsized":"2024-09-13T17:42:07.294395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:10:21.939396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331677+0000","last_change":"2024-09-13T17:42:07.317665+0000","last_active":"2024-09-13T17:42:07.331677+0000","last_peered":"2024-09-13T17:42:07.331677+0000","last_clean":"2024-09-13T17:42:07.331677+0000","last_became_active":"2024-09-13T17:42:07.317401+0000","last_became_peered":"2024-09-13T17:42:07.317401+0000","last_unstale":"2024-09-13T17:42:07.331677+0000","last_undegraded":"2024-09-13T17:42:07.331677+0000","last_fullsized":"2024-09-13T17:42:07.331677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:12:55.460668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.295483+0000","last_change":"2024-09-13T17:42:06.313488+0000","last_active":"2024-09-13T17:42:07.295483+0000","last_peered":"2024-09-13T17:42:07.295483+0000","last_clean":"2024-09-13T17:42:07.295483+0000","last_became_active":"2024-09-13T17:42:06.313241+0000","last_became_peered":"2024-09-13T17:42:06.313241+0000","last_unstale":"2024-09-13T17:42:07.295483+0000","last_undegraded":"2024-09-13T17:42:07.295483+0000","last_fullsized":"2024-09-13T17:42:07.295483+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:48:48.364768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.335898+0000","last_change":"2024-09-13T17:42:07.333605+0000","last_active":"2024-09-13T17:42:07.335898+0000","last_peered":"2024-09-13T17:42:07.335898+0000","last_clean":"2024-09-13T17:42:07.335898+0000","last_became_active":"2024-09-13T17:42:07.332509+0000","last_became_peered":"2024-09-13T17:42:07.332509+0000","last_unstale":"2024-09-13T17:42:07.335898+0000","last_undegraded":"2024-09-13T17:42:07.335898+0000","last_fullsized":"2024-09-13T17:42:07.335898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:26:11.888585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.295399+0000","last_change":"2024-09-13T17:42:06.310629+0000","last_active":"2024-09-13T17:42:07.295399+0000","last_peered":"2024-09-13T17:42:07.295399+0000","last_clean":"2024-09-13T17:42:07.295399+0000","last_became_active":"2024-09-13T17:42:06.310103+0000","last_became_peered":"2024-09-13T17:42:06.310103+0000","last_unstale":"2024-09-13T17:42:07.295399+0000","last_undegraded":"2024-09-13T17:42:07.295399+0000","last_fullsized":"2024-09-13T17:42:07.295399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:42.121791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.340677+0000","last_change":"2024-09-13T17:42:07.335957+0000","last_active":"2024-09-13T17:42:07.340677+0000","last_peered":"2024-09-13T17:42:07.340677+0000","last_clean":"2024-09-13T17:42:07.340677+0000","last_became_active":"2024-09-13T17:42:07.331187+0000","last_became_peered":"2024-09-13T17:42:07.331187+0000","last_unstale":"2024-09-13T17:42:07.340677+0000","last_undegraded":"2024-09-13T17:42:07.340677+0000","last_fullsized":"2024-09-13T17:42:07.340677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:44.405426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":84,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.308327+0000","last_change":"2024-09-13T17:41:55.654539+0000","last_active":"2024-09-13T17:42:07.308327+0000","last_peered":"2024-09-13T17:42:07.308327+0000","last_clean":"2024-09-13T17:42:07.308327+0000","last_became_active":"2024-09-13T17:41:55.653536+0000","last_became_peered":"2024-09-13T17:41:55.653536+0000","last_unstale":"2024-09-13T17:42:07.308327+0000","last_undegraded":"2024-09-13T17:42:07.308327+0000","last_fullsized":"2024-09-13T17:42:07.308327+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:54.532965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:54.532965+0000","last_clean_scrub_stamp":"2024-09-13T17:41:54.532965+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:56.662588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.825620+0000","last_change":"2024-09-13T17:42:02.137356+0000","last_active":"2024-09-13T17:42:07.825620+0000","last_peered":"2024-09-13T17:42:07.825620+0000","last_clean":"2024-09-13T17:42:07.825620+0000","last_became_active":"2024-09-13T17:42:00.116534+0000","last_became_peered":"2024-09-13T17:42:00.116534+0000","last_unstale":"2024-09-13T17:42:07.825620+0000","last_undegraded":"2024-09-13T17:42:07.825620+0000","last_fullsized":"2024-09-13T17:42:07.825620+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:06.665067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000321799,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.294511+0000","last_change":"2024-09-13T17:42:06.312144+0000","last_active":"2024-09-13T17:42:07.294511+0000","last_peered":"2024-09-13T17:42:07.294511+0000","last_clean":"2024-09-13T17:42:07.294511+0000","last_became_active":"2024-09-13T17:42:06.311843+0000","last_became_peered":"2024-09-13T17:42:06.311843+0000","last_unstale":"2024-09-13T17:42:07.294511+0000","last_undegraded":"2024-09-13T17:42:07.294511+0000","last_fullsized":"2024-09-13T17:42:07.294511+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:16.479088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.332628+0000","last_change":"2024-09-13T17:42:07.329208+0000","last_active":"2024-09-13T17:42:07.332628+0000","last_peered":"2024-09-13T17:42:07.332628+0000","last_clean":"2024-09-13T17:42:07.332628+0000","last_became_active":"2024-09-13T17:42:07.328947+0000","last_became_peered":"2024-09-13T17:42:07.328947+0000","last_unstale":"2024-09-13T17:42:07.332628+0000","last_undegraded":"2024-09-13T17:42:07.332628+0000","last_fullsized":"2024-09-13T17:42:07.332628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:25:39.337142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.824598+0000","last_change":"2024-09-13T17:42:02.137373+0000","last_active":"2024-09-13T17:42:07.824598+0000","last_peered":"2024-09-13T17:42:07.824598+0000","last_clean":"2024-09-13T17:42:07.824598+0000","last_became_active":"2024-09-13T17:42:00.116542+0000","last_became_peered":"2024-09-13T17:42:00.116542+0000","last_unstale":"2024-09-13T17:42:07.824598+0000","last_undegraded":"2024-09-13T17:42:07.824598+0000","last_fullsized":"2024-09-13T17:42:07.824598+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:53.754563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035063,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.303137+0000","last_change":"2024-09-13T17:42:06.304286+0000","last_active":"2024-09-13T17:42:07.303137+0000","last_peered":"2024-09-13T17:42:07.303137+0000","last_clean":"2024-09-13T17:42:07.303137+0000","last_became_active":"2024-09-13T17:42:06.304053+0000","last_became_peered":"2024-09-13T17:42:06.304053+0000","last_unstale":"2024-09-13T17:42:07.303137+0000","last_undegraded":"2024-09-13T17:42:07.303137+0000","last_fullsized":"2024-09-13T17:42:07.303137+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:20:52.415835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338397+0000","last_change":"2024-09-13T17:42:07.326878+0000","last_active":"2024-09-13T17:42:07.338397+0000","last_peered":"2024-09-13T17:42:07.338397+0000","last_clean":"2024-09-13T17:42:07.338397+0000","last_became_active":"2024-09-13T17:42:07.326416+0000","last_became_peered":"2024-09-13T17:42:07.326416+0000","last_unstale":"2024-09-13T17:42:07.338397+0000","last_undegraded":"2024-09-13T17:42:07.338397+0000","last_fullsized":"2024-09-13T17:42:07.338397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:45.068616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.384334+0000","last_change":"2024-09-13T17:42:02.615831+0000","last_active":"2024-09-13T17:42:07.384334+0000","last_peered":"2024-09-13T17:42:07.384334+0000","last_clean":"2024-09-13T17:42:07.384334+0000","last_became_active":"2024-09-13T17:42:00.116427+0000","last_became_peered":"2024-09-13T17:42:00.116427+0000","last_unstale":"2024-09-13T17:42:07.384334+0000","last_undegraded":"2024-09-13T17:42:07.384334+0000","last_fullsized":"2024-09-13T17:42:07.384334+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:29:17.848348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000228423,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.310191+0000","last_change":"2024-09-13T17:42:06.302978+0000","last_active":"2024-09-13T17:42:07.310191+0000","last_peered":"2024-09-13T17:42:07.310191+0000","last_clean":"2024-09-13T17:42:07.310191+0000","last_became_active":"2024-09-13T17:42:06.302485+0000","last_became_peered":"2024-09-13T17:42:06.302485+0000","last_unstale":"2024-09-13T17:42:07.310191+0000","last_undegraded":"2024-09-13T17:42:07.310191+0000","last_fullsized":"2024-09-13T17:42:07.310191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:17:23.133953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.325491+0000","last_change":"2024-09-13T17:42:07.318090+0000","last_active":"2024-09-13T17:42:07.325491+0000","last_peered":"2024-09-13T17:42:07.325491+0000","last_clean":"2024-09-13T17:42:07.325491+0000","last_became_active":"2024-09-13T17:42:07.317337+0000","last_became_peered":"2024-09-13T17:42:07.317337+0000","last_unstale":"2024-09-13T17:42:07.325491+0000","last_undegraded":"2024-09-13T17:42:07.325491+0000","last_fullsized":"2024-09-13T17:42:07.325491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:02:50.060149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.824778+0000","last_change":"2024-09-13T17:42:02.138081+0000","last_active":"2024-09-13T17:42:07.824778+0000","last_peered":"2024-09-13T17:42:07.824778+0000","last_clean":"2024-09-13T17:42:07.824778+0000","last_became_active":"2024-09-13T17:42:00.116465+0000","last_became_peered":"2024-09-13T17:42:00.116465+0000","last_unstale":"2024-09-13T17:42:07.824778+0000","last_undegraded":"2024-09-13T17:42:07.824778+0000","last_fullsized":"2024-09-13T17:42:07.824778+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:40:30.342531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001026573,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.308940+0000","last_change":"2024-09-13T17:42:06.304017+0000","last_active":"2024-09-13T17:42:07.308940+0000","last_peered":"2024-09-13T17:42:07.308940+0000","last_clean":"2024-09-13T17:42:07.308940+0000","last_became_active":"2024-09-13T17:42:06.303764+0000","last_became_peered":"2024-09-13T17:42:06.303764+0000","last_unstale":"2024-09-13T17:42:07.308940+0000","last_undegraded":"2024-09-13T17:42:07.308940+0000","last_fullsized":"2024-09-13T17:42:07.308940+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:20:51.407014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.325010+0000","last_change":"2024-09-13T17:42:07.314438+0000","last_active":"2024-09-13T17:42:07.325010+0000","last_peered":"2024-09-13T17:42:07.325010+0000","last_clean":"2024-09-13T17:42:07.325010+0000","last_became_active":"2024-09-13T17:42:07.314196+0000","last_became_peered":"2024-09-13T17:42:07.314196+0000","last_unstale":"2024-09-13T17:42:07.325010+0000","last_undegraded":"2024-09-13T17:42:07.325010+0000","last_fullsized":"2024-09-13T17:42:07.325010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:33:57.543704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.298253+0000","last_change":"2024-09-13T17:42:06.304819+0000","last_active":"2024-09-13T17:42:07.298253+0000","last_peered":"2024-09-13T17:42:07.298253+0000","last_clean":"2024-09-13T17:42:07.298253+0000","last_became_active":"2024-09-13T17:42:06.304542+0000","last_became_peered":"2024-09-13T17:42:06.304542+0000","last_unstale":"2024-09-13T17:42:07.298253+0000","last_undegraded":"2024-09-13T17:42:07.298253+0000","last_fullsized":"2024-09-13T17:42:07.298253+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:10:57.127180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T17:42:08.701451+0000","last_change":"2024-09-13T17:42:02.135135+0000","last_active":"2024-09-13T17:42:08.701451+0000","last_peered":"2024-09-13T17:42:08.701451+0000","last_clean":"2024-09-13T17:42:08.701451+0000","last_became_active":"2024-09-13T17:42:00.364240+0000","last_became_peered":"2024-09-13T17:42:00.364240+0000","last_unstale":"2024-09-13T17:42:08.701451+0000","last_undegraded":"2024-09-13T17:42:08.701451+0000","last_fullsized":"2024-09-13T17:42:08.701451+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:31:29.382006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575944,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.342655+0000","last_change":"2024-09-13T17:42:07.338051+0000","last_active":"2024-09-13T17:42:07.342655+0000","last_peered":"2024-09-13T17:42:07.342655+0000","last_clean":"2024-09-13T17:42:07.342655+0000","last_became_active":"2024-09-13T17:42:07.337334+0000","last_became_peered":"2024-09-13T17:42:07.337334+0000","last_unstale":"2024-09-13T17:42:07.342655+0000","last_undegraded":"2024-09-13T17:42:07.342655+0000","last_fullsized":"2024-09-13T17:42:07.342655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:45.068616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.302622+0000","last_change":"2024-09-13T17:42:06.300680+0000","last_active":"2024-09-13T17:42:07.302622+0000","last_peered":"2024-09-13T17:42:07.302622+0000","last_clean":"2024-09-13T17:42:07.302622+0000","last_became_active":"2024-09-13T17:42:06.299518+0000","last_became_peered":"2024-09-13T17:42:06.299518+0000","last_unstale":"2024-09-13T17:42:07.302622+0000","last_undegraded":"2024-09-13T17:42:07.302622+0000","last_fullsized":"2024-09-13T17:42:07.302622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:20:52.415835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.825916+0000","last_change":"2024-09-13T17:42:02.138136+0000","last_active":"2024-09-13T17:42:07.825916+0000","last_peered":"2024-09-13T17:42:07.825916+0000","last_clean":"2024-09-13T17:42:07.825916+0000","last_became_active":"2024-09-13T17:42:00.364233+0000","last_became_peered":"2024-09-13T17:42:00.364233+0000","last_unstale":"2024-09-13T17:42:07.825916+0000","last_undegraded":"2024-09-13T17:42:07.825916+0000","last_fullsized":"2024-09-13T17:42:07.825916+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:54.234194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010757130000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.322084+0000","last_change":"2024-09-13T17:42:07.315025+0000","last_active":"2024-09-13T17:42:07.322084+0000","last_peered":"2024-09-13T17:42:07.322084+0000","last_clean":"2024-09-13T17:42:07.322084+0000","last_became_active":"2024-09-13T17:42:07.314074+0000","last_became_peered":"2024-09-13T17:42:07.314074+0000","last_unstale":"2024-09-13T17:42:07.322084+0000","last_undegraded":"2024-09-13T17:42:07.322084+0000","last_fullsized":"2024-09-13T17:42:07.322084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:25.632002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.340850+0000","last_change":"2024-09-13T17:42:06.310475+0000","last_active":"2024-09-13T17:42:07.340850+0000","last_peered":"2024-09-13T17:42:07.340850+0000","last_clean":"2024-09-13T17:42:07.340850+0000","last_became_active":"2024-09-13T17:42:06.310230+0000","last_became_peered":"2024-09-13T17:42:06.310230+0000","last_unstale":"2024-09-13T17:42:07.340850+0000","last_undegraded":"2024-09-13T17:42:07.340850+0000","last_fullsized":"2024-09-13T17:42:07.340850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:30:29.499136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T17:42:08.701455+0000","last_change":"2024-09-13T17:42:02.134963+0000","last_active":"2024-09-13T17:42:08.701455+0000","last_peered":"2024-09-13T17:42:08.701455+0000","last_clean":"2024-09-13T17:42:08.701455+0000","last_became_active":"2024-09-13T17:42:00.364233+0000","last_became_peered":"2024-09-13T17:42:00.364233+0000","last_unstale":"2024-09-13T17:42:08.701455+0000","last_undegraded":"2024-09-13T17:42:08.701455+0000","last_fullsized":"2024-09-13T17:42:08.701455+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:24:31.191067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393223,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.334115+0000","last_change":"2024-09-13T17:42:07.324146+0000","last_active":"2024-09-13T17:42:07.334115+0000","last_peered":"2024-09-13T17:42:07.334115+0000","last_clean":"2024-09-13T17:42:07.334115+0000","last_became_active":"2024-09-13T17:42:07.323120+0000","last_became_peered":"2024-09-13T17:42:07.323120+0000","last_unstale":"2024-09-13T17:42:07.334115+0000","last_undegraded":"2024-09-13T17:42:07.334115+0000","last_fullsized":"2024-09-13T17:42:07.334115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:20:26.946513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.310176+0000","last_change":"2024-09-13T17:42:06.305416+0000","last_active":"2024-09-13T17:42:07.310176+0000","last_peered":"2024-09-13T17:42:07.310176+0000","last_clean":"2024-09-13T17:42:07.310176+0000","last_became_active":"2024-09-13T17:42:06.304782+0000","last_became_peered":"2024-09-13T17:42:06.304782+0000","last_unstale":"2024-09-13T17:42:07.310176+0000","last_undegraded":"2024-09-13T17:42:07.310176+0000","last_fullsized":"2024-09-13T17:42:07.310176+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:08:01.276478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.825679+0000","last_change":"2024-09-13T17:42:02.138362+0000","last_active":"2024-09-13T17:42:07.825679+0000","last_peered":"2024-09-13T17:42:07.825679+0000","last_clean":"2024-09-13T17:42:07.825679+0000","last_became_active":"2024-09-13T17:42:00.364214+0000","last_became_peered":"2024-09-13T17:42:00.364214+0000","last_unstale":"2024-09-13T17:42:07.825679+0000","last_undegraded":"2024-09-13T17:42:07.825679+0000","last_fullsized":"2024-09-13T17:42:07.825679+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:33.976642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012776910000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.334811+0000","last_change":"2024-09-13T17:42:07.332265+0000","last_active":"2024-09-13T17:42:07.334811+0000","last_peered":"2024-09-13T17:42:07.334811+0000","last_clean":"2024-09-13T17:42:07.334811+0000","last_became_active":"2024-09-13T17:42:07.331834+0000","last_became_peered":"2024-09-13T17:42:07.331834+0000","last_unstale":"2024-09-13T17:42:07.334811+0000","last_undegraded":"2024-09-13T17:42:07.334811+0000","last_fullsized":"2024-09-13T17:42:07.334811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:52:45.662321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.335827+0000","last_change":"2024-09-13T17:42:07.333367+0000","last_active":"2024-09-13T17:42:07.335827+0000","last_peered":"2024-09-13T17:42:07.335827+0000","last_clean":"2024-09-13T17:42:07.335827+0000","last_became_active":"2024-09-13T17:42:07.333132+0000","last_became_peered":"2024-09-13T17:42:07.333132+0000","last_unstale":"2024-09-13T17:42:07.335827+0000","last_undegraded":"2024-09-13T17:42:07.335827+0000","last_fullsized":"2024-09-13T17:42:07.335827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:43.125769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"19'6","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.342572+0000","last_change":"2024-09-13T17:42:06.305919+0000","last_active":"2024-09-13T17:42:07.342572+0000","last_peered":"2024-09-13T17:42:07.342572+0000","last_clean":"2024-09-13T17:42:07.342572+0000","last_became_active":"2024-09-13T17:42:06.305663+0000","last_became_peered":"2024-09-13T17:42:06.305663+0000","last_unstale":"2024-09-13T17:42:07.342572+0000","last_undegraded":"2024-09-13T17:42:07.342572+0000","last_fullsized":"2024-09-13T17:42:07.342572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:24.628023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.340404+0000","last_change":"2024-09-13T17:42:07.330596+0000","last_active":"2024-09-13T17:42:07.340404+0000","last_peered":"2024-09-13T17:42:07.340404+0000","last_clean":"2024-09-13T17:42:07.340404+0000","last_became_active":"2024-09-13T17:42:07.330349+0000","last_became_peered":"2024-09-13T17:42:07.330349+0000","last_unstale":"2024-09-13T17:42:07.340404+0000","last_undegraded":"2024-09-13T17:42:07.340404+0000","last_fullsized":"2024-09-13T17:42:07.340404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:43.466051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.334688+0000","last_change":"2024-09-13T17:42:06.309276+0000","last_active":"2024-09-13T17:42:07.334688+0000","last_peered":"2024-09-13T17:42:07.334688+0000","last_clean":"2024-09-13T17:42:07.334688+0000","last_became_active":"2024-09-13T17:42:06.307311+0000","last_became_peered":"2024-09-13T17:42:06.307311+0000","last_unstale":"2024-09-13T17:42:07.334688+0000","last_undegraded":"2024-09-13T17:42:07.334688+0000","last_fullsized":"2024-09-13T17:42:07.334688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:31:06.878681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.299687+0000","last_change":"2024-09-13T17:42:06.301289+0000","last_active":"2024-09-13T17:42:07.299687+0000","last_peered":"2024-09-13T17:42:07.299687+0000","last_clean":"2024-09-13T17:42:07.299687+0000","last_became_active":"2024-09-13T17:42:06.301040+0000","last_became_peered":"2024-09-13T17:42:06.301040+0000","last_unstale":"2024-09-13T17:42:07.299687+0000","last_undegraded":"2024-09-13T17:42:07.299687+0000","last_fullsized":"2024-09-13T17:42:07.299687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:49:37.251198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331603+0000","last_change":"2024-09-13T17:42:07.321515+0000","last_active":"2024-09-13T17:42:07.331603+0000","last_peered":"2024-09-13T17:42:07.331603+0000","last_clean":"2024-09-13T17:42:07.331603+0000","last_became_active":"2024-09-13T17:42:07.312719+0000","last_became_peered":"2024-09-13T17:42:07.312719+0000","last_unstale":"2024-09-13T17:42:07.331603+0000","last_undegraded":"2024-09-13T17:42:07.331603+0000","last_fullsized":"2024-09-13T17:42:07.331603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:29:48.289765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.308212+0000","last_change":"2024-09-13T17:42:06.307317+0000","last_active":"2024-09-13T17:42:07.308212+0000","last_peered":"2024-09-13T17:42:07.308212+0000","last_clean":"2024-09-13T17:42:07.308212+0000","last_became_active":"2024-09-13T17:42:06.307087+0000","last_became_peered":"2024-09-13T17:42:06.307087+0000","last_unstale":"2024-09-13T17:42:07.308212+0000","last_undegraded":"2024-09-13T17:42:07.308212+0000","last_fullsized":"2024-09-13T17:42:07.308212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:47:16.711770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.339794+0000","last_change":"2024-09-13T17:42:07.333996+0000","last_active":"2024-09-13T17:42:07.339794+0000","last_peered":"2024-09-13T17:42:07.339794+0000","last_clean":"2024-09-13T17:42:07.339794+0000","last_became_active":"2024-09-13T17:42:07.329707+0000","last_became_peered":"2024-09-13T17:42:07.329707+0000","last_unstale":"2024-09-13T17:42:07.339794+0000","last_undegraded":"2024-09-13T17:42:07.339794+0000","last_fullsized":"2024-09-13T17:42:07.339794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:01:57.881608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331148+0000","last_change":"2024-09-13T17:42:07.314437+0000","last_active":"2024-09-13T17:42:07.331148+0000","last_peered":"2024-09-13T17:42:07.331148+0000","last_clean":"2024-09-13T17:42:07.331148+0000","last_became_active":"2024-09-13T17:42:07.314194+0000","last_became_peered":"2024-09-13T17:42:07.314194+0000","last_unstale":"2024-09-13T17:42:07.331148+0000","last_undegraded":"2024-09-13T17:42:07.331148+0000","last_fullsized":"2024-09-13T17:42:07.331148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:42:37.705801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.295292+0000","last_change":"2024-09-13T17:42:06.308176+0000","last_active":"2024-09-13T17:42:07.295292+0000","last_peered":"2024-09-13T17:42:07.295292+0000","last_clean":"2024-09-13T17:42:07.295292+0000","last_became_active":"2024-09-13T17:42:06.307906+0000","last_became_peered":"2024-09-13T17:42:06.307906+0000","last_unstale":"2024-09-13T17:42:07.295292+0000","last_undegraded":"2024-09-13T17:42:07.295292+0000","last_fullsized":"2024-09-13T17:42:07.295292+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:26:10.884606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338381+0000","last_change":"2024-09-13T17:42:07.326636+0000","last_active":"2024-09-13T17:42:07.338381+0000","last_peered":"2024-09-13T17:42:07.338381+0000","last_clean":"2024-09-13T17:42:07.338381+0000","last_became_active":"2024-09-13T17:42:07.326146+0000","last_became_peered":"2024-09-13T17:42:07.326146+0000","last_unstale":"2024-09-13T17:42:07.338381+0000","last_undegraded":"2024-09-13T17:42:07.338381+0000","last_fullsized":"2024-09-13T17:42:07.338381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:22:40.880345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.298760+0000","last_change":"2024-09-13T17:42:06.312966+0000","last_active":"2024-09-13T17:42:07.298760+0000","last_peered":"2024-09-13T17:42:07.298760+0000","last_clean":"2024-09-13T17:42:07.298760+0000","last_became_active":"2024-09-13T17:42:06.312726+0000","last_became_peered":"2024-09-13T17:42:06.312726+0000","last_unstale":"2024-09-13T17:42:07.298760+0000","last_undegraded":"2024-09-13T17:42:07.298760+0000","last_fullsized":"2024-09-13T17:42:07.298760+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:59.621392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331661+0000","last_change":"2024-09-13T17:42:07.315788+0000","last_active":"2024-09-13T17:42:07.331661+0000","last_peered":"2024-09-13T17:42:07.331661+0000","last_clean":"2024-09-13T17:42:07.331661+0000","last_became_active":"2024-09-13T17:42:07.312751+0000","last_became_peered":"2024-09-13T17:42:07.312751+0000","last_unstale":"2024-09-13T17:42:07.331661+0000","last_undegraded":"2024-09-13T17:42:07.331661+0000","last_fullsized":"2024-09-13T17:42:07.331661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:24:19.756257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.326108+0000","last_change":"2024-09-13T17:42:06.303521+0000","last_active":"2024-09-13T17:42:07.326108+0000","last_peered":"2024-09-13T17:42:07.326108+0000","last_clean":"2024-09-13T17:42:07.326108+0000","last_became_active":"2024-09-13T17:42:06.303171+0000","last_became_peered":"2024-09-13T17:42:06.303171+0000","last_unstale":"2024-09-13T17:42:07.326108+0000","last_undegraded":"2024-09-13T17:42:07.326108+0000","last_fullsized":"2024-09-13T17:42:07.326108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:36:02.512243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.327020+0000","last_change":"2024-09-13T17:42:07.316113+0000","last_active":"2024-09-13T17:42:07.327020+0000","last_peered":"2024-09-13T17:42:07.327020+0000","last_clean":"2024-09-13T17:42:07.327020+0000","last_became_active":"2024-09-13T17:42:07.315546+0000","last_became_peered":"2024-09-13T17:42:07.315546+0000","last_unstale":"2024-09-13T17:42:07.327020+0000","last_undegraded":"2024-09-13T17:42:07.327020+0000","last_fullsized":"2024-09-13T17:42:07.327020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:30:30.503115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.335532+0000","last_change":"2024-09-13T17:42:06.305594+0000","last_active":"2024-09-13T17:42:07.335532+0000","last_peered":"2024-09-13T17:42:07.335532+0000","last_clean":"2024-09-13T17:42:07.335532+0000","last_became_active":"2024-09-13T17:42:06.305330+0000","last_became_peered":"2024-09-13T17:42:06.305330+0000","last_unstale":"2024-09-13T17:42:07.335532+0000","last_undegraded":"2024-09-13T17:42:07.335532+0000","last_fullsized":"2024-09-13T17:42:07.335532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:17:23.133953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.335793+0000","last_change":"2024-09-13T17:42:07.333147+0000","last_active":"2024-09-13T17:42:07.335793+0000","last_peered":"2024-09-13T17:42:07.335793+0000","last_clean":"2024-09-13T17:42:07.335793+0000","last_became_active":"2024-09-13T17:42:07.331637+0000","last_became_peered":"2024-09-13T17:42:07.331637+0000","last_unstale":"2024-09-13T17:42:07.335793+0000","last_undegraded":"2024-09-13T17:42:07.335793+0000","last_fullsized":"2024-09-13T17:42:07.335793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:25.632002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.328050+0000","last_change":"2024-09-13T17:42:06.312084+0000","last_active":"2024-09-13T17:42:07.328050+0000","last_peered":"2024-09-13T17:42:07.328050+0000","last_clean":"2024-09-13T17:42:07.328050+0000","last_became_active":"2024-09-13T17:42:06.311848+0000","last_became_peered":"2024-09-13T17:42:06.311848+0000","last_unstale":"2024-09-13T17:42:07.328050+0000","last_undegraded":"2024-09-13T17:42:07.328050+0000","last_fullsized":"2024-09-13T17:42:07.328050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:32.379434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.335857+0000","last_change":"2024-09-13T17:42:07.333648+0000","last_active":"2024-09-13T17:42:07.335857+0000","last_peered":"2024-09-13T17:42:07.335857+0000","last_clean":"2024-09-13T17:42:07.335857+0000","last_became_active":"2024-09-13T17:42:07.332563+0000","last_became_peered":"2024-09-13T17:42:07.332563+0000","last_unstale":"2024-09-13T17:42:07.335857+0000","last_undegraded":"2024-09-13T17:42:07.335857+0000","last_fullsized":"2024-09-13T17:42:07.335857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:06.934321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331832+0000","last_change":"2024-09-13T17:42:06.308815+0000","last_active":"2024-09-13T17:42:07.331832+0000","last_peered":"2024-09-13T17:42:07.331832+0000","last_clean":"2024-09-13T17:42:07.331832+0000","last_became_active":"2024-09-13T17:42:06.308407+0000","last_became_peered":"2024-09-13T17:42:06.308407+0000","last_unstale":"2024-09-13T17:42:07.331832+0000","last_undegraded":"2024-09-13T17:42:07.331832+0000","last_fullsized":"2024-09-13T17:42:07.331832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:19.237563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331281+0000","last_change":"2024-09-13T17:42:07.315582+0000","last_active":"2024-09-13T17:42:07.331281+0000","last_peered":"2024-09-13T17:42:07.331281+0000","last_clean":"2024-09-13T17:42:07.331281+0000","last_became_active":"2024-09-13T17:42:07.312277+0000","last_became_peered":"2024-09-13T17:42:07.312277+0000","last_unstale":"2024-09-13T17:42:07.331281+0000","last_undegraded":"2024-09-13T17:42:07.331281+0000","last_fullsized":"2024-09-13T17:42:07.331281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:59:50.392102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338321+0000","last_change":"2024-09-13T17:42:06.312228+0000","last_active":"2024-09-13T17:42:07.338321+0000","last_peered":"2024-09-13T17:42:07.338321+0000","last_clean":"2024-09-13T17:42:07.338321+0000","last_became_active":"2024-09-13T17:42:06.311930+0000","last_became_peered":"2024-09-13T17:42:06.311930+0000","last_unstale":"2024-09-13T17:42:07.338321+0000","last_undegraded":"2024-09-13T17:42:07.338321+0000","last_fullsized":"2024-09-13T17:42:07.338321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:16.993105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.330780+0000","last_change":"2024-09-13T17:42:07.330018+0000","last_active":"2024-09-13T17:42:07.330780+0000","last_peered":"2024-09-13T17:42:07.330780+0000","last_clean":"2024-09-13T17:42:07.330780+0000","last_became_active":"2024-09-13T17:42:07.329763+0000","last_became_peered":"2024-09-13T17:42:07.329763+0000","last_unstale":"2024-09-13T17:42:07.330780+0000","last_undegraded":"2024-09-13T17:42:07.330780+0000","last_fullsized":"2024-09-13T17:42:07.330780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:38:02.719445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.337820+0000","last_change":"2024-09-13T17:42:06.305609+0000","last_active":"2024-09-13T17:42:07.337820+0000","last_peered":"2024-09-13T17:42:07.337820+0000","last_clean":"2024-09-13T17:42:07.337820+0000","last_became_active":"2024-09-13T17:42:06.304878+0000","last_became_peered":"2024-09-13T17:42:06.304878+0000","last_unstale":"2024-09-13T17:42:07.337820+0000","last_undegraded":"2024-09-13T17:42:07.337820+0000","last_fullsized":"2024-09-13T17:42:07.337820+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:32.379434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297556,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074284,"statfs":{"total":96636764160,"available":96332066816,"internally_reserved":0,"allocated":200704,"data_stored":57550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]}]},{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297540,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074300,"statfs":{"total":96636764160,"available":96332083200,"internally_reserved":0,"allocated":184320,"data_stored":57337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]}]},{"osd":1,"up_from":9,"seq":38654705672,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298128,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073712,"statfs":{"total":96636764160,"available":96331481088,"internally_reserved":0,"allocated":782336,"data_stored":646834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]}]},{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298120,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073720,"statfs":{"total":96636764160,"available":96331489280,"internally_reserved":0,"allocated":782336,"data_stored":646869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:42:14.968 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:42:15.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.095+0000 7f47ea094640 1 Processor -- start 2024-09-13T17:42:15.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.095+0000 7f47ea094640 1 -- start start 2024-09-13T17:42:15.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.095+0000 7f47ea094640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f47e4128330 0x7f47e4128700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:15.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.095+0000 7f47ea094640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f47e412bf50 0x7f47e4133b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:15.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.095+0000 7f47ea094640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47e4097be0 con 0x7f47e4128c40 2024-09-13T17:42:15.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.095+0000 7f47ea094640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e4098e70 con 0x7f47e4128330 2024-09-13T17:42:15.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.095+0000 7f47ea094640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f47e4098310 con 0x7f47e412bf50 2024-09-13T17:42:15.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.096+0000 7f47e37fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f47e4128330 0x7f47e4128700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:15.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.096+0000 7f47e3fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f47e412bf50 0x7f47e4133b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:15.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.096+0000 7f47e2ffd640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f47e4128c40 0x7f47e412b840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:60884/0 (socket says 172.21.15.102:60884) 2024-09-13T17:42:15.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.096+0000 7f47e3fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f47e412bf50 0x7f47e4133b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:46676/0 (socket says 172.21.15.102:46676) 2024-09-13T17:42:15.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.096+0000 7f47e37fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f47e4128330 0x7f47e4128700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35434/0 (socket says 172.21.15.102:35434) 2024-09-13T17:42:15.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.096+0000 7f47e3fff640 1 -- 172.21.15.102:0/2029477076 learned_addr learned my addr 172.21.15.102:0/2029477076 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:15.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.096+0000 7f47e37fe640 1 -- 172.21.15.102:0/2029477076 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f47e412bf50 msgr2=0x7f47e4133b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:15.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.096+0000 7f47e37fe640 1 --2- 172.21.15.102:0/2029477076 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f47e412bf50 0x7f47e4133b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.096+0000 7f47e37fe640 1 -- 172.21.15.102:0/2029477076 >> v1:172.21.15.102:6789/0 conn(0x7f47e4128c40 legacy=0x7f47e412b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:15.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.096+0000 7f47e37fe640 1 -- 172.21.15.102:0/2029477076 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e40985d0 con 0x7f47e4128330 2024-09-13T17:42:15.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.096+0000 7f47e27fc640 1 -- 172.21.15.102:0/2029477076 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3765944649 0 0) 0x7f47e4097be0 con 0x7f47e4128c40 2024-09-13T17:42:15.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.097+0000 7f47e37fe640 1 --2- 172.21.15.102:0/2029477076 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f47e4128330 0x7f47e4128700 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f47d400bb20 tx=0x7f47d400b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52e19ec1e97685b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:15.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.097+0000 7f47e27fc640 1 -- 172.21.15.102:0/2029477076 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d40325d0 con 0x7f47e4128330 2024-09-13T17:42:15.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.097+0000 7f47e27fc640 1 -- 172.21.15.102:0/2029477076 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d4032730 con 0x7f47e4128330 2024-09-13T17:42:15.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.097+0000 7f47e27fc640 1 -- 172.21.15.102:0/2029477076 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d4032ad0 con 0x7f47e4128330 2024-09-13T17:42:15.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.097+0000 7f47ea094640 1 -- 172.21.15.102:0/2029477076 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f47e4128330 msgr2=0x7f47e4128700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:15.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.097+0000 7f47ea094640 1 --2- 172.21.15.102:0/2029477076 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f47e4128330 0x7f47e4128700 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f47d400bb20 tx=0x7f47d400b3b0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.097+0000 7f47ea094640 1 -- 172.21.15.102:0/2029477076 shutdown_connections 2024-09-13T17:42:15.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.097+0000 7f47ea094640 1 --2- 172.21.15.102:0/2029477076 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f47e412bf50 0x7f47e4133b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.097+0000 7f47ea094640 1 --2- 172.21.15.102:0/2029477076 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f47e4128330 0x7f47e4128700 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.097+0000 7f47ea094640 1 -- 172.21.15.102:0/2029477076 >> 172.21.15.102:0/2029477076 conn(0x7f47e409e820 msgr2=0x7f47e409ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:15.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.097+0000 7f47ea094640 1 -- 172.21.15.102:0/2029477076 shutdown_connections 2024-09-13T17:42:15.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.097+0000 7f47ea094640 1 -- 172.21.15.102:0/2029477076 wait complete. 2024-09-13T17:42:15.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.098+0000 7f47ea094640 1 Processor -- start 2024-09-13T17:42:15.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.098+0000 7f47ea094640 1 -- start start 2024-09-13T17:42:15.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.099+0000 7f47ea094640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f47e4128330 0x7f47e413a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:15.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.099+0000 7f47ea094640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47e4128c40 0x7f47e413ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:15.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.099+0000 7f47ea094640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f47e412bf50 0x7f47e4114e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:15.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.099+0000 7f47ea094640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e4117620 con 0x7f47e4128c40 2024-09-13T17:42:15.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.099+0000 7f47ea094640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e41178d0 con 0x7f47e412bf50 2024-09-13T17:42:15.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.099+0000 7f47ea094640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f47e4116750 con 0x7f47e4128330 2024-09-13T17:42:15.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.099+0000 7f47e37fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f47e4128330 0x7f47e413a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:15.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.099+0000 7f47e2ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47e4128c40 0x7f47e413ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:15.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.099+0000 7f47e3fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f47e412bf50 0x7f47e4114e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:15.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.099+0000 7f47e37fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f47e4128330 0x7f47e413a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:46690/0 (socket says 172.21.15.102:46690) 2024-09-13T17:42:15.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.099+0000 7f47e3fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f47e412bf50 0x7f47e4114e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35448/0 (socket says 172.21.15.102:35448) 2024-09-13T17:42:15.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.099+0000 7f47e37fe640 1 -- 172.21.15.102:0/735214036 learned_addr learned my addr 172.21.15.102:0/735214036 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:15.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.099+0000 7f47e37fe640 1 -- 172.21.15.102:0/735214036 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f47e412bf50 msgr2=0x7f47e4114e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:15.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.099+0000 7f47e37fe640 1 --2- 172.21.15.102:0/735214036 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f47e412bf50 0x7f47e4114e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.099+0000 7f47e37fe640 1 -- 172.21.15.102:0/735214036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47e4128c40 msgr2=0x7f47e413ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:15.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.099+0000 7f47e37fe640 1 --2- 172.21.15.102:0/735214036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47e4128c40 0x7f47e413ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.099+0000 7f47e37fe640 1 -- 172.21.15.102:0/735214036 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d400b630 con 0x7f47e4128330 2024-09-13T17:42:15.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.099+0000 7f47e3fff640 1 --2- 172.21.15.102:0/735214036 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f47e412bf50 0x7f47e4114e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:15.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.099+0000 7f47e2ffd640 1 --2- 172.21.15.102:0/735214036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47e4128c40 0x7f47e413ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:15.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.099+0000 7f47e37fe640 1 --2- 172.21.15.102:0/735214036 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f47e4128330 0x7f47e413a8c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f47d400b3f0 tx=0x7f47d403a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:15.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.099+0000 7f47cb7fe640 1 -- 172.21.15.102:0/735214036 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d4033460 con 0x7f47e4128330 2024-09-13T17:42:15.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.099+0000 7f47cb7fe640 1 -- 172.21.15.102:0/735214036 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d40335c0 con 0x7f47e4128330 2024-09-13T17:42:15.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.099+0000 7f47ea094640 1 -- 172.21.15.102:0/735214036 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47e413a680 con 0x7f47e4128330 2024-09-13T17:42:15.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.099+0000 7f47cb7fe640 1 -- 172.21.15.102:0/735214036 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d4033960 con 0x7f47e4128330 2024-09-13T17:42:15.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.100+0000 7f47ea094640 1 -- 172.21.15.102:0/735214036 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e413b340 con 0x7f47e4128330 2024-09-13T17:42:15.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.100+0000 7f47cb7fe640 1 -- 172.21.15.102:0/735214036 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f47d4031070 con 0x7f47e4128330 2024-09-13T17:42:15.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.100+0000 7f47cb7fe640 1 -- 172.21.15.102:0/735214036 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7f47d4066d80 con 0x7f47e4128330 2024-09-13T17:42:15.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.100+0000 7f47ea094640 1 -- 172.21.15.102:0/735214036 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47e4064000 con 0x7f47e4128330 2024-09-13T17:42:15.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.105+0000 7f47cb7fe640 1 -- 172.21.15.102:0/735214036 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f47d4033cd0 con 0x7f47e4128330 2024-09-13T17:42:15.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.255+0000 7f47ea094640 1 -- 172.21.15.102:0/735214036 --> v1:172.21.15.102:6816/34937 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f47e4116100 con 0x7f47d0031160 2024-09-13T17:42:15.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.261+0000 7f47cb7fe640 1 -- 172.21.15.102:0/735214036 <== mgr.4101 v1:172.21.15.102:6816/34937 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+271380 (unknown 2965378022 0 909147157) 0x7f47e4116100 con 0x7f47d0031160 2024-09-13T17:42:15.265 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:42:15.267 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-09-13T17:42:15.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.264+0000 7f47ea094640 1 -- 172.21.15.102:0/735214036 >> v1:172.21.15.102:6816/34937 conn(0x7f47d0031160 legacy=0x7f47d0033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:15.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.264+0000 7f47ea094640 1 -- 172.21.15.102:0/735214036 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f47e4128330 msgr2=0x7f47e413a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:15.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.264+0000 7f47ea094640 1 --2- 172.21.15.102:0/735214036 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f47e4128330 0x7f47e413a8c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f47d400b3f0 tx=0x7f47d403a000 comp rx=0 tx=0).stop 2024-09-13T17:42:15.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.264+0000 7f47ea094640 1 -- 172.21.15.102:0/735214036 shutdown_connections 2024-09-13T17:42:15.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.264+0000 7f47ea094640 1 --2- 172.21.15.102:0/735214036 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f47e412bf50 0x7f47e4114e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.264+0000 7f47ea094640 1 --2- 172.21.15.102:0/735214036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47e4128c40 0x7f47e413ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.264+0000 7f47ea094640 1 --2- 172.21.15.102:0/735214036 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f47e4128330 0x7f47e413a8c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.264+0000 7f47ea094640 1 -- 172.21.15.102:0/735214036 >> 172.21.15.102:0/735214036 conn(0x7f47e409e820 msgr2=0x7f47e41273e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:15.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.264+0000 7f47ea094640 1 -- 172.21.15.102:0/735214036 shutdown_connections 2024-09-13T17:42:15.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.264+0000 7f47ea094640 1 -- 172.21.15.102:0/735214036 wait complete. 2024-09-13T17:42:15.276 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-13T17:42:14.413816+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592725,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":125,"num_write_kb":1121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":93,"ondisk_log_size":93,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1191344,"kb_used_data":1904,"kb_used_omap":6,"kb_used_meta":1189369,"kb_avail":376296016,"statfs":{"total":386547056640,"available":385327120384,"internally_reserved":0,"allocated":1949696,"data_stored":1408590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6816,"internal_metadata":1217914208},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001349"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.334754+0000","last_change":"2024-09-13T17:42:07.332154+0000","last_active":"2024-09-13T17:42:07.334754+0000","last_peered":"2024-09-13T17:42:07.334754+0000","last_clean":"2024-09-13T17:42:07.334754+0000","last_became_active":"2024-09-13T17:42:07.331874+0000","last_became_peered":"2024-09-13T17:42:07.331874+0000","last_unstale":"2024-09-13T17:42:07.334754+0000","last_undegraded":"2024-09-13T17:42:07.334754+0000","last_fullsized":"2024-09-13T17:42:07.334754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:10:22.943375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.329161+0000","last_change":"2024-09-13T17:42:07.326086+0000","last_active":"2024-09-13T17:42:07.329161+0000","last_peered":"2024-09-13T17:42:07.329161+0000","last_clean":"2024-09-13T17:42:07.329161+0000","last_became_active":"2024-09-13T17:42:07.325830+0000","last_became_peered":"2024-09-13T17:42:07.325830+0000","last_unstale":"2024-09-13T17:42:07.329161+0000","last_undegraded":"2024-09-13T17:42:07.329161+0000","last_fullsized":"2024-09-13T17:42:07.329161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:54.719512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338441+0000","last_change":"2024-09-13T17:42:07.326953+0000","last_active":"2024-09-13T17:42:07.338441+0000","last_peered":"2024-09-13T17:42:07.338441+0000","last_clean":"2024-09-13T17:42:07.338441+0000","last_became_active":"2024-09-13T17:42:07.326717+0000","last_became_peered":"2024-09-13T17:42:07.326717+0000","last_unstale":"2024-09-13T17:42:07.338441+0000","last_undegraded":"2024-09-13T17:42:07.338441+0000","last_fullsized":"2024-09-13T17:42:07.338441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:20.647404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338424+0000","last_change":"2024-09-13T17:42:07.326356+0000","last_active":"2024-09-13T17:42:07.338424+0000","last_peered":"2024-09-13T17:42:07.338424+0000","last_clean":"2024-09-13T17:42:07.338424+0000","last_became_active":"2024-09-13T17:42:07.326127+0000","last_became_peered":"2024-09-13T17:42:07.326127+0000","last_unstale":"2024-09-13T17:42:07.338424+0000","last_undegraded":"2024-09-13T17:42:07.338424+0000","last_fullsized":"2024-09-13T17:42:07.338424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:44.405426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338304+0000","last_change":"2024-09-13T17:42:07.330092+0000","last_active":"2024-09-13T17:42:07.338304+0000","last_peered":"2024-09-13T17:42:07.338304+0000","last_clean":"2024-09-13T17:42:07.338304+0000","last_became_active":"2024-09-13T17:42:07.327658+0000","last_became_peered":"2024-09-13T17:42:07.327658+0000","last_unstale":"2024-09-13T17:42:07.338304+0000","last_undegraded":"2024-09-13T17:42:07.338304+0000","last_fullsized":"2024-09-13T17:42:07.338304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:24:02.920430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.324569+0000","last_change":"2024-09-13T17:42:07.303609+0000","last_active":"2024-09-13T17:42:07.324569+0000","last_peered":"2024-09-13T17:42:07.324569+0000","last_clean":"2024-09-13T17:42:07.324569+0000","last_became_active":"2024-09-13T17:42:07.303384+0000","last_became_peered":"2024-09-13T17:42:07.303384+0000","last_unstale":"2024-09-13T17:42:07.324569+0000","last_undegraded":"2024-09-13T17:42:07.324569+0000","last_fullsized":"2024-09-13T17:42:07.324569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:43:56.000592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.326264+0000","last_change":"2024-09-13T17:42:07.307630+0000","last_active":"2024-09-13T17:42:07.326264+0000","last_peered":"2024-09-13T17:42:07.326264+0000","last_clean":"2024-09-13T17:42:07.326264+0000","last_became_active":"2024-09-13T17:42:07.307382+0000","last_became_peered":"2024-09-13T17:42:07.307382+0000","last_unstale":"2024-09-13T17:42:07.326264+0000","last_undegraded":"2024-09-13T17:42:07.326264+0000","last_fullsized":"2024-09-13T17:42:07.326264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:26:32.846287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.326676+0000","last_change":"2024-09-13T17:42:07.320736+0000","last_active":"2024-09-13T17:42:07.326676+0000","last_peered":"2024-09-13T17:42:07.326676+0000","last_clean":"2024-09-13T17:42:07.326676+0000","last_became_active":"2024-09-13T17:42:07.319801+0000","last_became_peered":"2024-09-13T17:42:07.319801+0000","last_unstale":"2024-09-13T17:42:07.326676+0000","last_undegraded":"2024-09-13T17:42:07.326676+0000","last_fullsized":"2024-09-13T17:42:07.326676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:28:32.030397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.339799+0000","last_change":"2024-09-13T17:42:07.335208+0000","last_active":"2024-09-13T17:42:07.339799+0000","last_peered":"2024-09-13T17:42:07.339799+0000","last_clean":"2024-09-13T17:42:07.339799+0000","last_became_active":"2024-09-13T17:42:07.334956+0000","last_became_peered":"2024-09-13T17:42:07.334956+0000","last_unstale":"2024-09-13T17:42:07.339799+0000","last_undegraded":"2024-09-13T17:42:07.339799+0000","last_fullsized":"2024-09-13T17:42:07.339799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:59:53.544086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.325383+0000","last_change":"2024-09-13T17:42:07.317057+0000","last_active":"2024-09-13T17:42:07.325383+0000","last_peered":"2024-09-13T17:42:07.325383+0000","last_clean":"2024-09-13T17:42:07.325383+0000","last_became_active":"2024-09-13T17:42:07.316429+0000","last_became_peered":"2024-09-13T17:42:07.316429+0000","last_unstale":"2024-09-13T17:42:07.325383+0000","last_undegraded":"2024-09-13T17:42:07.325383+0000","last_fullsized":"2024-09-13T17:42:07.325383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:59:53.544086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.325590+0000","last_change":"2024-09-13T17:42:07.321106+0000","last_active":"2024-09-13T17:42:07.325590+0000","last_peered":"2024-09-13T17:42:07.325590+0000","last_clean":"2024-09-13T17:42:07.325590+0000","last_became_active":"2024-09-13T17:42:07.320827+0000","last_became_peered":"2024-09-13T17:42:07.320827+0000","last_unstale":"2024-09-13T17:42:07.325590+0000","last_undegraded":"2024-09-13T17:42:07.325590+0000","last_fullsized":"2024-09-13T17:42:07.325590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:34:34.794230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.334785+0000","last_change":"2024-09-13T17:42:07.331865+0000","last_active":"2024-09-13T17:42:07.334785+0000","last_peered":"2024-09-13T17:42:07.334785+0000","last_clean":"2024-09-13T17:42:07.334785+0000","last_became_active":"2024-09-13T17:42:07.331419+0000","last_became_peered":"2024-09-13T17:42:07.331419+0000","last_unstale":"2024-09-13T17:42:07.334785+0000","last_undegraded":"2024-09-13T17:42:07.334785+0000","last_fullsized":"2024-09-13T17:42:07.334785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:48.155823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.335811+0000","last_change":"2024-09-13T17:42:07.333296+0000","last_active":"2024-09-13T17:42:07.335811+0000","last_peered":"2024-09-13T17:42:07.335811+0000","last_clean":"2024-09-13T17:42:07.335811+0000","last_became_active":"2024-09-13T17:42:07.332898+0000","last_became_peered":"2024-09-13T17:42:07.332898+0000","last_unstale":"2024-09-13T17:42:07.335811+0000","last_undegraded":"2024-09-13T17:42:07.335811+0000","last_fullsized":"2024-09-13T17:42:07.335811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:20.241542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338928+0000","last_change":"2024-09-13T17:42:07.324771+0000","last_active":"2024-09-13T17:42:07.338928+0000","last_peered":"2024-09-13T17:42:07.338928+0000","last_clean":"2024-09-13T17:42:07.338928+0000","last_became_active":"2024-09-13T17:42:07.323820+0000","last_became_peered":"2024-09-13T17:42:07.323820+0000","last_unstale":"2024-09-13T17:42:07.338928+0000","last_undegraded":"2024-09-13T17:42:07.338928+0000","last_fullsized":"2024-09-13T17:42:07.338928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:24:02.920430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.337730+0000","last_change":"2024-09-13T17:42:07.325827+0000","last_active":"2024-09-13T17:42:07.337730+0000","last_peered":"2024-09-13T17:42:07.337730+0000","last_clean":"2024-09-13T17:42:07.337730+0000","last_became_active":"2024-09-13T17:42:07.325589+0000","last_became_peered":"2024-09-13T17:42:07.325589+0000","last_unstale":"2024-09-13T17:42:07.337730+0000","last_undegraded":"2024-09-13T17:42:07.337730+0000","last_fullsized":"2024-09-13T17:42:07.337730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:43.466051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.334093+0000","last_change":"2024-09-13T17:42:07.315478+0000","last_active":"2024-09-13T17:42:07.334093+0000","last_peered":"2024-09-13T17:42:07.334093+0000","last_clean":"2024-09-13T17:42:07.334093+0000","last_became_active":"2024-09-13T17:42:07.315218+0000","last_became_peered":"2024-09-13T17:42:07.315218+0000","last_unstale":"2024-09-13T17:42:07.334093+0000","last_undegraded":"2024-09-13T17:42:07.334093+0000","last_fullsized":"2024-09-13T17:42:07.334093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:06:44.563276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338279+0000","last_change":"2024-09-13T17:42:07.328192+0000","last_active":"2024-09-13T17:42:07.338279+0000","last_peered":"2024-09-13T17:42:07.338279+0000","last_clean":"2024-09-13T17:42:07.338279+0000","last_became_active":"2024-09-13T17:42:07.327951+0000","last_became_peered":"2024-09-13T17:42:07.327951+0000","last_unstale":"2024-09-13T17:42:07.338279+0000","last_undegraded":"2024-09-13T17:42:07.338279+0000","last_fullsized":"2024-09-13T17:42:07.338279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:28:32.030397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.320769+0000","last_change":"2024-09-13T17:42:07.314816+0000","last_active":"2024-09-13T17:42:07.320769+0000","last_peered":"2024-09-13T17:42:07.320769+0000","last_clean":"2024-09-13T17:42:07.320769+0000","last_became_active":"2024-09-13T17:42:07.314464+0000","last_became_peered":"2024-09-13T17:42:07.314464+0000","last_unstale":"2024-09-13T17:42:07.320769+0000","last_undegraded":"2024-09-13T17:42:07.320769+0000","last_fullsized":"2024-09-13T17:42:07.320769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:17.483066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.335387+0000","last_change":"2024-09-13T17:42:07.332269+0000","last_active":"2024-09-13T17:42:07.335387+0000","last_peered":"2024-09-13T17:42:07.335387+0000","last_clean":"2024-09-13T17:42:07.335387+0000","last_became_active":"2024-09-13T17:42:07.332040+0000","last_became_peered":"2024-09-13T17:42:07.332040+0000","last_unstale":"2024-09-13T17:42:07.335387+0000","last_undegraded":"2024-09-13T17:42:07.335387+0000","last_fullsized":"2024-09-13T17:42:07.335387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:48:49.368747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.340627+0000","last_change":"2024-09-13T17:42:07.335425+0000","last_active":"2024-09-13T17:42:07.340627+0000","last_peered":"2024-09-13T17:42:07.340627+0000","last_clean":"2024-09-13T17:42:07.340627+0000","last_became_active":"2024-09-13T17:42:07.335173+0000","last_became_peered":"2024-09-13T17:42:07.335173+0000","last_unstale":"2024-09-13T17:42:07.340627+0000","last_undegraded":"2024-09-13T17:42:07.340627+0000","last_fullsized":"2024-09-13T17:42:07.340627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:00:10.355609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.340514+0000","last_change":"2024-09-13T17:42:07.331183+0000","last_active":"2024-09-13T17:42:07.340514+0000","last_peered":"2024-09-13T17:42:07.340514+0000","last_clean":"2024-09-13T17:42:07.340514+0000","last_became_active":"2024-09-13T17:42:07.330936+0000","last_became_peered":"2024-09-13T17:42:07.330936+0000","last_unstale":"2024-09-13T17:42:07.340514+0000","last_undegraded":"2024-09-13T17:42:07.340514+0000","last_fullsized":"2024-09-13T17:42:07.340514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:42:38.505262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.337694+0000","last_change":"2024-09-13T17:42:07.325877+0000","last_active":"2024-09-13T17:42:07.337694+0000","last_peered":"2024-09-13T17:42:07.337694+0000","last_clean":"2024-09-13T17:42:07.337694+0000","last_became_active":"2024-09-13T17:42:07.325657+0000","last_became_peered":"2024-09-13T17:42:07.325657+0000","last_unstale":"2024-09-13T17:42:07.337694+0000","last_undegraded":"2024-09-13T17:42:07.337694+0000","last_fullsized":"2024-09-13T17:42:07.337694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:48.663311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331637+0000","last_change":"2024-09-13T17:42:07.322059+0000","last_active":"2024-09-13T17:42:07.331637+0000","last_peered":"2024-09-13T17:42:07.331637+0000","last_clean":"2024-09-13T17:42:07.331637+0000","last_became_active":"2024-09-13T17:42:07.312536+0000","last_became_peered":"2024-09-13T17:42:07.312536+0000","last_unstale":"2024-09-13T17:42:07.331637+0000","last_undegraded":"2024-09-13T17:42:07.331637+0000","last_fullsized":"2024-09-13T17:42:07.331637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:14:08.780968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.340774+0000","last_change":"2024-09-13T17:42:07.336557+0000","last_active":"2024-09-13T17:42:07.340774+0000","last_peered":"2024-09-13T17:42:07.340774+0000","last_clean":"2024-09-13T17:42:07.340774+0000","last_became_active":"2024-09-13T17:42:07.335820+0000","last_became_peered":"2024-09-13T17:42:07.335820+0000","last_unstale":"2024-09-13T17:42:07.340774+0000","last_undegraded":"2024-09-13T17:42:07.340774+0000","last_fullsized":"2024-09-13T17:42:07.340774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:20.647404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.342633+0000","last_change":"2024-09-13T17:42:07.337695+0000","last_active":"2024-09-13T17:42:07.342633+0000","last_peered":"2024-09-13T17:42:07.342633+0000","last_clean":"2024-09-13T17:42:07.342633+0000","last_became_active":"2024-09-13T17:42:07.337419+0000","last_became_peered":"2024-09-13T17:42:07.337419+0000","last_unstale":"2024-09-13T17:42:07.342633+0000","last_undegraded":"2024-09-13T17:42:07.342633+0000","last_fullsized":"2024-09-13T17:42:07.342633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:22:40.880345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.325365+0000","last_change":"2024-09-13T17:42:07.315008+0000","last_active":"2024-09-13T17:42:07.325365+0000","last_peered":"2024-09-13T17:42:07.325365+0000","last_clean":"2024-09-13T17:42:07.325365+0000","last_became_active":"2024-09-13T17:42:07.313477+0000","last_became_peered":"2024-09-13T17:42:07.313477+0000","last_unstale":"2024-09-13T17:42:07.325365+0000","last_undegraded":"2024-09-13T17:42:07.325365+0000","last_fullsized":"2024-09-13T17:42:07.325365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:25.632002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331694+0000","last_change":"2024-09-13T17:42:07.317949+0000","last_active":"2024-09-13T17:42:07.331694+0000","last_peered":"2024-09-13T17:42:07.331694+0000","last_clean":"2024-09-13T17:42:07.331694+0000","last_became_active":"2024-09-13T17:42:07.317720+0000","last_became_peered":"2024-09-13T17:42:07.317720+0000","last_unstale":"2024-09-13T17:42:07.331694+0000","last_undegraded":"2024-09-13T17:42:07.331694+0000","last_fullsized":"2024-09-13T17:42:07.331694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:31.378417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.324990+0000","last_change":"2024-09-13T17:42:07.313528+0000","last_active":"2024-09-13T17:42:07.324990+0000","last_peered":"2024-09-13T17:42:07.324990+0000","last_clean":"2024-09-13T17:42:07.324990+0000","last_became_active":"2024-09-13T17:42:07.313092+0000","last_became_peered":"2024-09-13T17:42:07.313092+0000","last_unstale":"2024-09-13T17:42:07.324990+0000","last_undegraded":"2024-09-13T17:42:07.324990+0000","last_fullsized":"2024-09-13T17:42:07.324990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:48:49.368747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331153+0000","last_change":"2024-09-13T17:42:07.314816+0000","last_active":"2024-09-13T17:42:07.331153+0000","last_peered":"2024-09-13T17:42:07.331153+0000","last_clean":"2024-09-13T17:42:07.331153+0000","last_became_active":"2024-09-13T17:42:07.314574+0000","last_became_peered":"2024-09-13T17:42:07.314574+0000","last_unstale":"2024-09-13T17:42:07.331153+0000","last_undegraded":"2024-09-13T17:42:07.331153+0000","last_fullsized":"2024-09-13T17:42:07.331153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:03:35.976183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338349+0000","last_change":"2024-09-13T17:42:07.328522+0000","last_active":"2024-09-13T17:42:07.338349+0000","last_peered":"2024-09-13T17:42:07.338349+0000","last_clean":"2024-09-13T17:42:07.338349+0000","last_became_active":"2024-09-13T17:42:07.327893+0000","last_became_peered":"2024-09-13T17:42:07.327893+0000","last_unstale":"2024-09-13T17:42:07.338349+0000","last_undegraded":"2024-09-13T17:42:07.338349+0000","last_fullsized":"2024-09-13T17:42:07.338349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:02:50.060149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331620+0000","last_change":"2024-09-13T17:42:07.321900+0000","last_active":"2024-09-13T17:42:07.331620+0000","last_peered":"2024-09-13T17:42:07.331620+0000","last_clean":"2024-09-13T17:42:07.331620+0000","last_became_active":"2024-09-13T17:42:07.319230+0000","last_became_peered":"2024-09-13T17:42:07.319230+0000","last_unstale":"2024-09-13T17:42:07.331620+0000","last_undegraded":"2024-09-13T17:42:07.331620+0000","last_fullsized":"2024-09-13T17:42:07.331620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:50:46.442172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.326655+0000","last_change":"2024-09-13T17:42:07.315846+0000","last_active":"2024-09-13T17:42:07.326655+0000","last_peered":"2024-09-13T17:42:07.326655+0000","last_clean":"2024-09-13T17:42:07.326655+0000","last_became_active":"2024-09-13T17:42:07.315513+0000","last_became_peered":"2024-09-13T17:42:07.315513+0000","last_unstale":"2024-09-13T17:42:07.326655+0000","last_undegraded":"2024-09-13T17:42:07.326655+0000","last_fullsized":"2024-09-13T17:42:07.326655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:56:21.343829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331104+0000","last_change":"2024-09-13T17:42:07.315400+0000","last_active":"2024-09-13T17:42:07.331104+0000","last_peered":"2024-09-13T17:42:07.331104+0000","last_clean":"2024-09-13T17:42:07.331104+0000","last_became_active":"2024-09-13T17:42:07.315131+0000","last_became_peered":"2024-09-13T17:42:07.315131+0000","last_unstale":"2024-09-13T17:42:07.331104+0000","last_undegraded":"2024-09-13T17:42:07.331104+0000","last_fullsized":"2024-09-13T17:42:07.331104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:21:39.335847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338775+0000","last_change":"2024-09-13T17:42:06.308504+0000","last_active":"2024-09-13T17:42:07.338775+0000","last_peered":"2024-09-13T17:42:07.338775+0000","last_clean":"2024-09-13T17:42:07.338775+0000","last_became_active":"2024-09-13T17:42:06.308199+0000","last_became_peered":"2024-09-13T17:42:06.308199+0000","last_unstale":"2024-09-13T17:42:07.338775+0000","last_undegraded":"2024-09-13T17:42:07.338775+0000","last_fullsized":"2024-09-13T17:42:07.338775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:05.930342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331918+0000","last_change":"2024-09-13T17:42:07.322597+0000","last_active":"2024-09-13T17:42:07.331918+0000","last_peered":"2024-09-13T17:42:07.331918+0000","last_clean":"2024-09-13T17:42:07.331918+0000","last_became_active":"2024-09-13T17:42:07.321945+0000","last_became_peered":"2024-09-13T17:42:07.321945+0000","last_unstale":"2024-09-13T17:42:07.331918+0000","last_undegraded":"2024-09-13T17:42:07.331918+0000","last_fullsized":"2024-09-13T17:42:07.331918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:03:28.132876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.301168+0000","last_change":"2024-09-13T17:42:06.306367+0000","last_active":"2024-09-13T17:42:07.301168+0000","last_peered":"2024-09-13T17:42:07.301168+0000","last_clean":"2024-09-13T17:42:07.301168+0000","last_became_active":"2024-09-13T17:42:06.306146+0000","last_became_peered":"2024-09-13T17:42:06.306146+0000","last_unstale":"2024-09-13T17:42:07.301168+0000","last_undegraded":"2024-09-13T17:42:07.301168+0000","last_fullsized":"2024-09-13T17:42:07.301168+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:45:01.750995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338414+0000","last_change":"2024-09-13T17:42:07.329528+0000","last_active":"2024-09-13T17:42:07.338414+0000","last_peered":"2024-09-13T17:42:07.338414+0000","last_clean":"2024-09-13T17:42:07.338414+0000","last_became_active":"2024-09-13T17:42:07.329296+0000","last_became_peered":"2024-09-13T17:42:07.329296+0000","last_unstale":"2024-09-13T17:42:07.338414+0000","last_undegraded":"2024-09-13T17:42:07.338414+0000","last_fullsized":"2024-09-13T17:42:07.338414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:33:57.543704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.294808+0000","last_change":"2024-09-13T17:42:06.312798+0000","last_active":"2024-09-13T17:42:07.294808+0000","last_peered":"2024-09-13T17:42:07.294808+0000","last_clean":"2024-09-13T17:42:07.294808+0000","last_became_active":"2024-09-13T17:42:06.312388+0000","last_became_peered":"2024-09-13T17:42:06.312388+0000","last_unstale":"2024-09-13T17:42:07.294808+0000","last_undegraded":"2024-09-13T17:42:07.294808+0000","last_fullsized":"2024-09-13T17:42:07.294808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:56:20.339850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331165+0000","last_change":"2024-09-13T17:42:07.315290+0000","last_active":"2024-09-13T17:42:07.331165+0000","last_peered":"2024-09-13T17:42:07.331165+0000","last_clean":"2024-09-13T17:42:07.331165+0000","last_became_active":"2024-09-13T17:42:07.314670+0000","last_became_peered":"2024-09-13T17:42:07.314670+0000","last_unstale":"2024-09-13T17:42:07.331165+0000","last_undegraded":"2024-09-13T17:42:07.331165+0000","last_fullsized":"2024-09-13T17:42:07.331165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:00:24.969812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.340570+0000","last_change":"2024-09-13T17:42:06.305806+0000","last_active":"2024-09-13T17:42:07.340570+0000","last_peered":"2024-09-13T17:42:07.340570+0000","last_clean":"2024-09-13T17:42:07.340570+0000","last_became_active":"2024-09-13T17:42:06.305576+0000","last_became_peered":"2024-09-13T17:42:06.305576+0000","last_unstale":"2024-09-13T17:42:07.340570+0000","last_undegraded":"2024-09-13T17:42:07.340570+0000","last_fullsized":"2024-09-13T17:42:07.340570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:47:16.711770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.337092+0000","last_change":"2024-09-13T17:42:06.304106+0000","last_active":"2024-09-13T17:42:07.337092+0000","last_peered":"2024-09-13T17:42:07.337092+0000","last_clean":"2024-09-13T17:42:07.337092+0000","last_became_active":"2024-09-13T17:42:06.303845+0000","last_became_peered":"2024-09-13T17:42:06.303845+0000","last_unstale":"2024-09-13T17:42:07.337092+0000","last_undegraded":"2024-09-13T17:42:07.337092+0000","last_fullsized":"2024-09-13T17:42:07.337092+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:36:02.512243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.334525+0000","last_change":"2024-09-13T17:42:07.331147+0000","last_active":"2024-09-13T17:42:07.334525+0000","last_peered":"2024-09-13T17:42:07.334525+0000","last_clean":"2024-09-13T17:42:07.334525+0000","last_became_active":"2024-09-13T17:42:07.329386+0000","last_became_peered":"2024-09-13T17:42:07.329386+0000","last_unstale":"2024-09-13T17:42:07.334525+0000","last_undegraded":"2024-09-13T17:42:07.334525+0000","last_fullsized":"2024-09-13T17:42:07.334525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:59:21.210970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.321593+0000","last_change":"2024-09-13T17:42:06.304872+0000","last_active":"2024-09-13T17:42:07.321593+0000","last_peered":"2024-09-13T17:42:07.321593+0000","last_clean":"2024-09-13T17:42:07.321593+0000","last_became_active":"2024-09-13T17:42:06.304639+0000","last_became_peered":"2024-09-13T17:42:06.304639+0000","last_unstale":"2024-09-13T17:42:07.321593+0000","last_undegraded":"2024-09-13T17:42:07.321593+0000","last_fullsized":"2024-09-13T17:42:07.321593+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:40.147410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.340505+0000","last_change":"2024-09-13T17:42:07.331235+0000","last_active":"2024-09-13T17:42:07.340505+0000","last_peered":"2024-09-13T17:42:07.340505+0000","last_clean":"2024-09-13T17:42:07.340505+0000","last_became_active":"2024-09-13T17:42:07.330822+0000","last_became_peered":"2024-09-13T17:42:07.330822+0000","last_unstale":"2024-09-13T17:42:07.340505+0000","last_undegraded":"2024-09-13T17:42:07.340505+0000","last_fullsized":"2024-09-13T17:42:07.340505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:38:04.013092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.340781+0000","last_change":"2024-09-13T17:42:06.310319+0000","last_active":"2024-09-13T17:42:07.340781+0000","last_peered":"2024-09-13T17:42:07.340781+0000","last_clean":"2024-09-13T17:42:07.340781+0000","last_became_active":"2024-09-13T17:42:06.310085+0000","last_became_peered":"2024-09-13T17:42:06.310085+0000","last_unstale":"2024-09-13T17:42:07.340781+0000","last_undegraded":"2024-09-13T17:42:07.340781+0000","last_fullsized":"2024-09-13T17:42:07.340781+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:53.715533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.335838+0000","last_change":"2024-09-13T17:42:07.333488+0000","last_active":"2024-09-13T17:42:07.335838+0000","last_peered":"2024-09-13T17:42:07.335838+0000","last_clean":"2024-09-13T17:42:07.335838+0000","last_became_active":"2024-09-13T17:42:07.332200+0000","last_became_peered":"2024-09-13T17:42:07.332200+0000","last_unstale":"2024-09-13T17:42:07.335838+0000","last_undegraded":"2024-09-13T17:42:07.335838+0000","last_fullsized":"2024-09-13T17:42:07.335838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:52.666716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.303967+0000","last_change":"2024-09-13T17:42:06.308854+0000","last_active":"2024-09-13T17:42:07.303967+0000","last_peered":"2024-09-13T17:42:07.303967+0000","last_clean":"2024-09-13T17:42:07.303967+0000","last_became_active":"2024-09-13T17:42:06.307173+0000","last_became_peered":"2024-09-13T17:42:06.307173+0000","last_unstale":"2024-09-13T17:42:07.303967+0000","last_undegraded":"2024-09-13T17:42:07.303967+0000","last_fullsized":"2024-09-13T17:42:07.303967+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:59.621392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338229+0000","last_change":"2024-09-13T17:42:07.327431+0000","last_active":"2024-09-13T17:42:07.338229+0000","last_peered":"2024-09-13T17:42:07.338229+0000","last_clean":"2024-09-13T17:42:07.338229+0000","last_became_active":"2024-09-13T17:42:07.327202+0000","last_became_peered":"2024-09-13T17:42:07.327202+0000","last_unstale":"2024-09-13T17:42:07.338229+0000","last_undegraded":"2024-09-13T17:42:07.338229+0000","last_fullsized":"2024-09-13T17:42:07.338229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:42:38.505262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338801+0000","last_change":"2024-09-13T17:42:06.308795+0000","last_active":"2024-09-13T17:42:07.338801+0000","last_peered":"2024-09-13T17:42:07.338801+0000","last_clean":"2024-09-13T17:42:07.338801+0000","last_became_active":"2024-09-13T17:42:06.308557+0000","last_became_peered":"2024-09-13T17:42:06.308557+0000","last_unstale":"2024-09-13T17:42:07.338801+0000","last_undegraded":"2024-09-13T17:42:07.338801+0000","last_fullsized":"2024-09-13T17:42:07.338801+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:51.662738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338442+0000","last_change":"2024-09-13T17:42:07.327849+0000","last_active":"2024-09-13T17:42:07.338442+0000","last_peered":"2024-09-13T17:42:07.338442+0000","last_clean":"2024-09-13T17:42:07.338442+0000","last_became_active":"2024-09-13T17:42:07.327578+0000","last_became_peered":"2024-09-13T17:42:07.327578+0000","last_unstale":"2024-09-13T17:42:07.338442+0000","last_undegraded":"2024-09-13T17:42:07.338442+0000","last_fullsized":"2024-09-13T17:42:07.338442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:34:34.794230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.294395+0000","last_change":"2024-09-13T17:42:06.310708+0000","last_active":"2024-09-13T17:42:07.294395+0000","last_peered":"2024-09-13T17:42:07.294395+0000","last_clean":"2024-09-13T17:42:07.294395+0000","last_became_active":"2024-09-13T17:42:06.310476+0000","last_became_peered":"2024-09-13T17:42:06.310476+0000","last_unstale":"2024-09-13T17:42:07.294395+0000","last_undegraded":"2024-09-13T17:42:07.294395+0000","last_fullsized":"2024-09-13T17:42:07.294395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:10:21.939396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331677+0000","last_change":"2024-09-13T17:42:07.317665+0000","last_active":"2024-09-13T17:42:07.331677+0000","last_peered":"2024-09-13T17:42:07.331677+0000","last_clean":"2024-09-13T17:42:07.331677+0000","last_became_active":"2024-09-13T17:42:07.317401+0000","last_became_peered":"2024-09-13T17:42:07.317401+0000","last_unstale":"2024-09-13T17:42:07.331677+0000","last_undegraded":"2024-09-13T17:42:07.331677+0000","last_fullsized":"2024-09-13T17:42:07.331677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:12:55.460668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.295483+0000","last_change":"2024-09-13T17:42:06.313488+0000","last_active":"2024-09-13T17:42:07.295483+0000","last_peered":"2024-09-13T17:42:07.295483+0000","last_clean":"2024-09-13T17:42:07.295483+0000","last_became_active":"2024-09-13T17:42:06.313241+0000","last_became_peered":"2024-09-13T17:42:06.313241+0000","last_unstale":"2024-09-13T17:42:07.295483+0000","last_undegraded":"2024-09-13T17:42:07.295483+0000","last_fullsized":"2024-09-13T17:42:07.295483+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:48:48.364768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.335898+0000","last_change":"2024-09-13T17:42:07.333605+0000","last_active":"2024-09-13T17:42:07.335898+0000","last_peered":"2024-09-13T17:42:07.335898+0000","last_clean":"2024-09-13T17:42:07.335898+0000","last_became_active":"2024-09-13T17:42:07.332509+0000","last_became_peered":"2024-09-13T17:42:07.332509+0000","last_unstale":"2024-09-13T17:42:07.335898+0000","last_undegraded":"2024-09-13T17:42:07.335898+0000","last_fullsized":"2024-09-13T17:42:07.335898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:26:11.888585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.295399+0000","last_change":"2024-09-13T17:42:06.310629+0000","last_active":"2024-09-13T17:42:07.295399+0000","last_peered":"2024-09-13T17:42:07.295399+0000","last_clean":"2024-09-13T17:42:07.295399+0000","last_became_active":"2024-09-13T17:42:06.310103+0000","last_became_peered":"2024-09-13T17:42:06.310103+0000","last_unstale":"2024-09-13T17:42:07.295399+0000","last_undegraded":"2024-09-13T17:42:07.295399+0000","last_fullsized":"2024-09-13T17:42:07.295399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:42.121791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.340677+0000","last_change":"2024-09-13T17:42:07.335957+0000","last_active":"2024-09-13T17:42:07.340677+0000","last_peered":"2024-09-13T17:42:07.340677+0000","last_clean":"2024-09-13T17:42:07.340677+0000","last_became_active":"2024-09-13T17:42:07.331187+0000","last_became_peered":"2024-09-13T17:42:07.331187+0000","last_unstale":"2024-09-13T17:42:07.340677+0000","last_undegraded":"2024-09-13T17:42:07.340677+0000","last_fullsized":"2024-09-13T17:42:07.340677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:44.405426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":84,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.308327+0000","last_change":"2024-09-13T17:41:55.654539+0000","last_active":"2024-09-13T17:42:07.308327+0000","last_peered":"2024-09-13T17:42:07.308327+0000","last_clean":"2024-09-13T17:42:07.308327+0000","last_became_active":"2024-09-13T17:41:55.653536+0000","last_became_peered":"2024-09-13T17:41:55.653536+0000","last_unstale":"2024-09-13T17:42:07.308327+0000","last_undegraded":"2024-09-13T17:42:07.308327+0000","last_fullsized":"2024-09-13T17:42:07.308327+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:54.532965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:54.532965+0000","last_clean_scrub_stamp":"2024-09-13T17:41:54.532965+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:56.662588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.825620+0000","last_change":"2024-09-13T17:42:02.137356+0000","last_active":"2024-09-13T17:42:07.825620+0000","last_peered":"2024-09-13T17:42:07.825620+0000","last_clean":"2024-09-13T17:42:07.825620+0000","last_became_active":"2024-09-13T17:42:00.116534+0000","last_became_peered":"2024-09-13T17:42:00.116534+0000","last_unstale":"2024-09-13T17:42:07.825620+0000","last_undegraded":"2024-09-13T17:42:07.825620+0000","last_fullsized":"2024-09-13T17:42:07.825620+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:06.665067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000321799,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.294511+0000","last_change":"2024-09-13T17:42:06.312144+0000","last_active":"2024-09-13T17:42:07.294511+0000","last_peered":"2024-09-13T17:42:07.294511+0000","last_clean":"2024-09-13T17:42:07.294511+0000","last_became_active":"2024-09-13T17:42:06.311843+0000","last_became_peered":"2024-09-13T17:42:06.311843+0000","last_unstale":"2024-09-13T17:42:07.294511+0000","last_undegraded":"2024-09-13T17:42:07.294511+0000","last_fullsized":"2024-09-13T17:42:07.294511+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:16.479088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.332628+0000","last_change":"2024-09-13T17:42:07.329208+0000","last_active":"2024-09-13T17:42:07.332628+0000","last_peered":"2024-09-13T17:42:07.332628+0000","last_clean":"2024-09-13T17:42:07.332628+0000","last_became_active":"2024-09-13T17:42:07.328947+0000","last_became_peered":"2024-09-13T17:42:07.328947+0000","last_unstale":"2024-09-13T17:42:07.332628+0000","last_undegraded":"2024-09-13T17:42:07.332628+0000","last_fullsized":"2024-09-13T17:42:07.332628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:25:39.337142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.824598+0000","last_change":"2024-09-13T17:42:02.137373+0000","last_active":"2024-09-13T17:42:07.824598+0000","last_peered":"2024-09-13T17:42:07.824598+0000","last_clean":"2024-09-13T17:42:07.824598+0000","last_became_active":"2024-09-13T17:42:00.116542+0000","last_became_peered":"2024-09-13T17:42:00.116542+0000","last_unstale":"2024-09-13T17:42:07.824598+0000","last_undegraded":"2024-09-13T17:42:07.824598+0000","last_fullsized":"2024-09-13T17:42:07.824598+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:53.754563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035063,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.303137+0000","last_change":"2024-09-13T17:42:06.304286+0000","last_active":"2024-09-13T17:42:07.303137+0000","last_peered":"2024-09-13T17:42:07.303137+0000","last_clean":"2024-09-13T17:42:07.303137+0000","last_became_active":"2024-09-13T17:42:06.304053+0000","last_became_peered":"2024-09-13T17:42:06.304053+0000","last_unstale":"2024-09-13T17:42:07.303137+0000","last_undegraded":"2024-09-13T17:42:07.303137+0000","last_fullsized":"2024-09-13T17:42:07.303137+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:20:52.415835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338397+0000","last_change":"2024-09-13T17:42:07.326878+0000","last_active":"2024-09-13T17:42:07.338397+0000","last_peered":"2024-09-13T17:42:07.338397+0000","last_clean":"2024-09-13T17:42:07.338397+0000","last_became_active":"2024-09-13T17:42:07.326416+0000","last_became_peered":"2024-09-13T17:42:07.326416+0000","last_unstale":"2024-09-13T17:42:07.338397+0000","last_undegraded":"2024-09-13T17:42:07.338397+0000","last_fullsized":"2024-09-13T17:42:07.338397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:45.068616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.384334+0000","last_change":"2024-09-13T17:42:02.615831+0000","last_active":"2024-09-13T17:42:07.384334+0000","last_peered":"2024-09-13T17:42:07.384334+0000","last_clean":"2024-09-13T17:42:07.384334+0000","last_became_active":"2024-09-13T17:42:00.116427+0000","last_became_peered":"2024-09-13T17:42:00.116427+0000","last_unstale":"2024-09-13T17:42:07.384334+0000","last_undegraded":"2024-09-13T17:42:07.384334+0000","last_fullsized":"2024-09-13T17:42:07.384334+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:29:17.848348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000228423,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.310191+0000","last_change":"2024-09-13T17:42:06.302978+0000","last_active":"2024-09-13T17:42:07.310191+0000","last_peered":"2024-09-13T17:42:07.310191+0000","last_clean":"2024-09-13T17:42:07.310191+0000","last_became_active":"2024-09-13T17:42:06.302485+0000","last_became_peered":"2024-09-13T17:42:06.302485+0000","last_unstale":"2024-09-13T17:42:07.310191+0000","last_undegraded":"2024-09-13T17:42:07.310191+0000","last_fullsized":"2024-09-13T17:42:07.310191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:17:23.133953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.325491+0000","last_change":"2024-09-13T17:42:07.318090+0000","last_active":"2024-09-13T17:42:07.325491+0000","last_peered":"2024-09-13T17:42:07.325491+0000","last_clean":"2024-09-13T17:42:07.325491+0000","last_became_active":"2024-09-13T17:42:07.317337+0000","last_became_peered":"2024-09-13T17:42:07.317337+0000","last_unstale":"2024-09-13T17:42:07.325491+0000","last_undegraded":"2024-09-13T17:42:07.325491+0000","last_fullsized":"2024-09-13T17:42:07.325491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:02:50.060149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.824778+0000","last_change":"2024-09-13T17:42:02.138081+0000","last_active":"2024-09-13T17:42:07.824778+0000","last_peered":"2024-09-13T17:42:07.824778+0000","last_clean":"2024-09-13T17:42:07.824778+0000","last_became_active":"2024-09-13T17:42:00.116465+0000","last_became_peered":"2024-09-13T17:42:00.116465+0000","last_unstale":"2024-09-13T17:42:07.824778+0000","last_undegraded":"2024-09-13T17:42:07.824778+0000","last_fullsized":"2024-09-13T17:42:07.824778+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:40:30.342531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001026573,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.308940+0000","last_change":"2024-09-13T17:42:06.304017+0000","last_active":"2024-09-13T17:42:07.308940+0000","last_peered":"2024-09-13T17:42:07.308940+0000","last_clean":"2024-09-13T17:42:07.308940+0000","last_became_active":"2024-09-13T17:42:06.303764+0000","last_became_peered":"2024-09-13T17:42:06.303764+0000","last_unstale":"2024-09-13T17:42:07.308940+0000","last_undegraded":"2024-09-13T17:42:07.308940+0000","last_fullsized":"2024-09-13T17:42:07.308940+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:20:51.407014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.325010+0000","last_change":"2024-09-13T17:42:07.314438+0000","last_active":"2024-09-13T17:42:07.325010+0000","last_peered":"2024-09-13T17:42:07.325010+0000","last_clean":"2024-09-13T17:42:07.325010+0000","last_became_active":"2024-09-13T17:42:07.314196+0000","last_became_peered":"2024-09-13T17:42:07.314196+0000","last_unstale":"2024-09-13T17:42:07.325010+0000","last_undegraded":"2024-09-13T17:42:07.325010+0000","last_fullsized":"2024-09-13T17:42:07.325010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:33:57.543704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.298253+0000","last_change":"2024-09-13T17:42:06.304819+0000","last_active":"2024-09-13T17:42:07.298253+0000","last_peered":"2024-09-13T17:42:07.298253+0000","last_clean":"2024-09-13T17:42:07.298253+0000","last_became_active":"2024-09-13T17:42:06.304542+0000","last_became_peered":"2024-09-13T17:42:06.304542+0000","last_unstale":"2024-09-13T17:42:07.298253+0000","last_undegraded":"2024-09-13T17:42:07.298253+0000","last_fullsized":"2024-09-13T17:42:07.298253+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:10:57.127180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T17:42:08.701451+0000","last_change":"2024-09-13T17:42:02.135135+0000","last_active":"2024-09-13T17:42:08.701451+0000","last_peered":"2024-09-13T17:42:08.701451+0000","last_clean":"2024-09-13T17:42:08.701451+0000","last_became_active":"2024-09-13T17:42:00.364240+0000","last_became_peered":"2024-09-13T17:42:00.364240+0000","last_unstale":"2024-09-13T17:42:08.701451+0000","last_undegraded":"2024-09-13T17:42:08.701451+0000","last_fullsized":"2024-09-13T17:42:08.701451+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:31:29.382006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575944,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.342655+0000","last_change":"2024-09-13T17:42:07.338051+0000","last_active":"2024-09-13T17:42:07.342655+0000","last_peered":"2024-09-13T17:42:07.342655+0000","last_clean":"2024-09-13T17:42:07.342655+0000","last_became_active":"2024-09-13T17:42:07.337334+0000","last_became_peered":"2024-09-13T17:42:07.337334+0000","last_unstale":"2024-09-13T17:42:07.342655+0000","last_undegraded":"2024-09-13T17:42:07.342655+0000","last_fullsized":"2024-09-13T17:42:07.342655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:45.068616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.302622+0000","last_change":"2024-09-13T17:42:06.300680+0000","last_active":"2024-09-13T17:42:07.302622+0000","last_peered":"2024-09-13T17:42:07.302622+0000","last_clean":"2024-09-13T17:42:07.302622+0000","last_became_active":"2024-09-13T17:42:06.299518+0000","last_became_peered":"2024-09-13T17:42:06.299518+0000","last_unstale":"2024-09-13T17:42:07.302622+0000","last_undegraded":"2024-09-13T17:42:07.302622+0000","last_fullsized":"2024-09-13T17:42:07.302622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:20:52.415835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.825916+0000","last_change":"2024-09-13T17:42:02.138136+0000","last_active":"2024-09-13T17:42:07.825916+0000","last_peered":"2024-09-13T17:42:07.825916+0000","last_clean":"2024-09-13T17:42:07.825916+0000","last_became_active":"2024-09-13T17:42:00.364233+0000","last_became_peered":"2024-09-13T17:42:00.364233+0000","last_unstale":"2024-09-13T17:42:07.825916+0000","last_undegraded":"2024-09-13T17:42:07.825916+0000","last_fullsized":"2024-09-13T17:42:07.825916+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:54.234194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010757130000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.322084+0000","last_change":"2024-09-13T17:42:07.315025+0000","last_active":"2024-09-13T17:42:07.322084+0000","last_peered":"2024-09-13T17:42:07.322084+0000","last_clean":"2024-09-13T17:42:07.322084+0000","last_became_active":"2024-09-13T17:42:07.314074+0000","last_became_peered":"2024-09-13T17:42:07.314074+0000","last_unstale":"2024-09-13T17:42:07.322084+0000","last_undegraded":"2024-09-13T17:42:07.322084+0000","last_fullsized":"2024-09-13T17:42:07.322084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:25.632002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.340850+0000","last_change":"2024-09-13T17:42:06.310475+0000","last_active":"2024-09-13T17:42:07.340850+0000","last_peered":"2024-09-13T17:42:07.340850+0000","last_clean":"2024-09-13T17:42:07.340850+0000","last_became_active":"2024-09-13T17:42:06.310230+0000","last_became_peered":"2024-09-13T17:42:06.310230+0000","last_unstale":"2024-09-13T17:42:07.340850+0000","last_undegraded":"2024-09-13T17:42:07.340850+0000","last_fullsized":"2024-09-13T17:42:07.340850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:30:29.499136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T17:42:08.701455+0000","last_change":"2024-09-13T17:42:02.134963+0000","last_active":"2024-09-13T17:42:08.701455+0000","last_peered":"2024-09-13T17:42:08.701455+0000","last_clean":"2024-09-13T17:42:08.701455+0000","last_became_active":"2024-09-13T17:42:00.364233+0000","last_became_peered":"2024-09-13T17:42:00.364233+0000","last_unstale":"2024-09-13T17:42:08.701455+0000","last_undegraded":"2024-09-13T17:42:08.701455+0000","last_fullsized":"2024-09-13T17:42:08.701455+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:24:31.191067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393223,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.334115+0000","last_change":"2024-09-13T17:42:07.324146+0000","last_active":"2024-09-13T17:42:07.334115+0000","last_peered":"2024-09-13T17:42:07.334115+0000","last_clean":"2024-09-13T17:42:07.334115+0000","last_became_active":"2024-09-13T17:42:07.323120+0000","last_became_peered":"2024-09-13T17:42:07.323120+0000","last_unstale":"2024-09-13T17:42:07.334115+0000","last_undegraded":"2024-09-13T17:42:07.334115+0000","last_fullsized":"2024-09-13T17:42:07.334115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:20:26.946513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.310176+0000","last_change":"2024-09-13T17:42:06.305416+0000","last_active":"2024-09-13T17:42:07.310176+0000","last_peered":"2024-09-13T17:42:07.310176+0000","last_clean":"2024-09-13T17:42:07.310176+0000","last_became_active":"2024-09-13T17:42:06.304782+0000","last_became_peered":"2024-09-13T17:42:06.304782+0000","last_unstale":"2024-09-13T17:42:07.310176+0000","last_undegraded":"2024-09-13T17:42:07.310176+0000","last_fullsized":"2024-09-13T17:42:07.310176+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:08:01.276478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.825679+0000","last_change":"2024-09-13T17:42:02.138362+0000","last_active":"2024-09-13T17:42:07.825679+0000","last_peered":"2024-09-13T17:42:07.825679+0000","last_clean":"2024-09-13T17:42:07.825679+0000","last_became_active":"2024-09-13T17:42:00.364214+0000","last_became_peered":"2024-09-13T17:42:00.364214+0000","last_unstale":"2024-09-13T17:42:07.825679+0000","last_undegraded":"2024-09-13T17:42:07.825679+0000","last_fullsized":"2024-09-13T17:42:07.825679+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:33.976642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012776910000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.334811+0000","last_change":"2024-09-13T17:42:07.332265+0000","last_active":"2024-09-13T17:42:07.334811+0000","last_peered":"2024-09-13T17:42:07.334811+0000","last_clean":"2024-09-13T17:42:07.334811+0000","last_became_active":"2024-09-13T17:42:07.331834+0000","last_became_peered":"2024-09-13T17:42:07.331834+0000","last_unstale":"2024-09-13T17:42:07.334811+0000","last_undegraded":"2024-09-13T17:42:07.334811+0000","last_fullsized":"2024-09-13T17:42:07.334811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:52:45.662321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.335827+0000","last_change":"2024-09-13T17:42:07.333367+0000","last_active":"2024-09-13T17:42:07.335827+0000","last_peered":"2024-09-13T17:42:07.335827+0000","last_clean":"2024-09-13T17:42:07.335827+0000","last_became_active":"2024-09-13T17:42:07.333132+0000","last_became_peered":"2024-09-13T17:42:07.333132+0000","last_unstale":"2024-09-13T17:42:07.335827+0000","last_undegraded":"2024-09-13T17:42:07.335827+0000","last_fullsized":"2024-09-13T17:42:07.335827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:43.125769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"19'6","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.342572+0000","last_change":"2024-09-13T17:42:06.305919+0000","last_active":"2024-09-13T17:42:07.342572+0000","last_peered":"2024-09-13T17:42:07.342572+0000","last_clean":"2024-09-13T17:42:07.342572+0000","last_became_active":"2024-09-13T17:42:06.305663+0000","last_became_peered":"2024-09-13T17:42:06.305663+0000","last_unstale":"2024-09-13T17:42:07.342572+0000","last_undegraded":"2024-09-13T17:42:07.342572+0000","last_fullsized":"2024-09-13T17:42:07.342572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:24.628023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.340404+0000","last_change":"2024-09-13T17:42:07.330596+0000","last_active":"2024-09-13T17:42:07.340404+0000","last_peered":"2024-09-13T17:42:07.340404+0000","last_clean":"2024-09-13T17:42:07.340404+0000","last_became_active":"2024-09-13T17:42:07.330349+0000","last_became_peered":"2024-09-13T17:42:07.330349+0000","last_unstale":"2024-09-13T17:42:07.340404+0000","last_undegraded":"2024-09-13T17:42:07.340404+0000","last_fullsized":"2024-09-13T17:42:07.340404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:43.466051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.334688+0000","last_change":"2024-09-13T17:42:06.309276+0000","last_active":"2024-09-13T17:42:07.334688+0000","last_peered":"2024-09-13T17:42:07.334688+0000","last_clean":"2024-09-13T17:42:07.334688+0000","last_became_active":"2024-09-13T17:42:06.307311+0000","last_became_peered":"2024-09-13T17:42:06.307311+0000","last_unstale":"2024-09-13T17:42:07.334688+0000","last_undegraded":"2024-09-13T17:42:07.334688+0000","last_fullsized":"2024-09-13T17:42:07.334688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:31:06.878681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.299687+0000","last_change":"2024-09-13T17:42:06.301289+0000","last_active":"2024-09-13T17:42:07.299687+0000","last_peered":"2024-09-13T17:42:07.299687+0000","last_clean":"2024-09-13T17:42:07.299687+0000","last_became_active":"2024-09-13T17:42:06.301040+0000","last_became_peered":"2024-09-13T17:42:06.301040+0000","last_unstale":"2024-09-13T17:42:07.299687+0000","last_undegraded":"2024-09-13T17:42:07.299687+0000","last_fullsized":"2024-09-13T17:42:07.299687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:49:37.251198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331603+0000","last_change":"2024-09-13T17:42:07.321515+0000","last_active":"2024-09-13T17:42:07.331603+0000","last_peered":"2024-09-13T17:42:07.331603+0000","last_clean":"2024-09-13T17:42:07.331603+0000","last_became_active":"2024-09-13T17:42:07.312719+0000","last_became_peered":"2024-09-13T17:42:07.312719+0000","last_unstale":"2024-09-13T17:42:07.331603+0000","last_undegraded":"2024-09-13T17:42:07.331603+0000","last_fullsized":"2024-09-13T17:42:07.331603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:29:48.289765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.308212+0000","last_change":"2024-09-13T17:42:06.307317+0000","last_active":"2024-09-13T17:42:07.308212+0000","last_peered":"2024-09-13T17:42:07.308212+0000","last_clean":"2024-09-13T17:42:07.308212+0000","last_became_active":"2024-09-13T17:42:06.307087+0000","last_became_peered":"2024-09-13T17:42:06.307087+0000","last_unstale":"2024-09-13T17:42:07.308212+0000","last_undegraded":"2024-09-13T17:42:07.308212+0000","last_fullsized":"2024-09-13T17:42:07.308212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:47:16.711770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.339794+0000","last_change":"2024-09-13T17:42:07.333996+0000","last_active":"2024-09-13T17:42:07.339794+0000","last_peered":"2024-09-13T17:42:07.339794+0000","last_clean":"2024-09-13T17:42:07.339794+0000","last_became_active":"2024-09-13T17:42:07.329707+0000","last_became_peered":"2024-09-13T17:42:07.329707+0000","last_unstale":"2024-09-13T17:42:07.339794+0000","last_undegraded":"2024-09-13T17:42:07.339794+0000","last_fullsized":"2024-09-13T17:42:07.339794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:01:57.881608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331148+0000","last_change":"2024-09-13T17:42:07.314437+0000","last_active":"2024-09-13T17:42:07.331148+0000","last_peered":"2024-09-13T17:42:07.331148+0000","last_clean":"2024-09-13T17:42:07.331148+0000","last_became_active":"2024-09-13T17:42:07.314194+0000","last_became_peered":"2024-09-13T17:42:07.314194+0000","last_unstale":"2024-09-13T17:42:07.331148+0000","last_undegraded":"2024-09-13T17:42:07.331148+0000","last_fullsized":"2024-09-13T17:42:07.331148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:42:37.705801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.295292+0000","last_change":"2024-09-13T17:42:06.308176+0000","last_active":"2024-09-13T17:42:07.295292+0000","last_peered":"2024-09-13T17:42:07.295292+0000","last_clean":"2024-09-13T17:42:07.295292+0000","last_became_active":"2024-09-13T17:42:06.307906+0000","last_became_peered":"2024-09-13T17:42:06.307906+0000","last_unstale":"2024-09-13T17:42:07.295292+0000","last_undegraded":"2024-09-13T17:42:07.295292+0000","last_fullsized":"2024-09-13T17:42:07.295292+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:26:10.884606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338381+0000","last_change":"2024-09-13T17:42:07.326636+0000","last_active":"2024-09-13T17:42:07.338381+0000","last_peered":"2024-09-13T17:42:07.338381+0000","last_clean":"2024-09-13T17:42:07.338381+0000","last_became_active":"2024-09-13T17:42:07.326146+0000","last_became_peered":"2024-09-13T17:42:07.326146+0000","last_unstale":"2024-09-13T17:42:07.338381+0000","last_undegraded":"2024-09-13T17:42:07.338381+0000","last_fullsized":"2024-09-13T17:42:07.338381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:22:40.880345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.298760+0000","last_change":"2024-09-13T17:42:06.312966+0000","last_active":"2024-09-13T17:42:07.298760+0000","last_peered":"2024-09-13T17:42:07.298760+0000","last_clean":"2024-09-13T17:42:07.298760+0000","last_became_active":"2024-09-13T17:42:06.312726+0000","last_became_peered":"2024-09-13T17:42:06.312726+0000","last_unstale":"2024-09-13T17:42:07.298760+0000","last_undegraded":"2024-09-13T17:42:07.298760+0000","last_fullsized":"2024-09-13T17:42:07.298760+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:59.621392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331661+0000","last_change":"2024-09-13T17:42:07.315788+0000","last_active":"2024-09-13T17:42:07.331661+0000","last_peered":"2024-09-13T17:42:07.331661+0000","last_clean":"2024-09-13T17:42:07.331661+0000","last_became_active":"2024-09-13T17:42:07.312751+0000","last_became_peered":"2024-09-13T17:42:07.312751+0000","last_unstale":"2024-09-13T17:42:07.331661+0000","last_undegraded":"2024-09-13T17:42:07.331661+0000","last_fullsized":"2024-09-13T17:42:07.331661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:24:19.756257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.326108+0000","last_change":"2024-09-13T17:42:06.303521+0000","last_active":"2024-09-13T17:42:07.326108+0000","last_peered":"2024-09-13T17:42:07.326108+0000","last_clean":"2024-09-13T17:42:07.326108+0000","last_became_active":"2024-09-13T17:42:06.303171+0000","last_became_peered":"2024-09-13T17:42:06.303171+0000","last_unstale":"2024-09-13T17:42:07.326108+0000","last_undegraded":"2024-09-13T17:42:07.326108+0000","last_fullsized":"2024-09-13T17:42:07.326108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:36:02.512243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.327020+0000","last_change":"2024-09-13T17:42:07.316113+0000","last_active":"2024-09-13T17:42:07.327020+0000","last_peered":"2024-09-13T17:42:07.327020+0000","last_clean":"2024-09-13T17:42:07.327020+0000","last_became_active":"2024-09-13T17:42:07.315546+0000","last_became_peered":"2024-09-13T17:42:07.315546+0000","last_unstale":"2024-09-13T17:42:07.327020+0000","last_undegraded":"2024-09-13T17:42:07.327020+0000","last_fullsized":"2024-09-13T17:42:07.327020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:30:30.503115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.335532+0000","last_change":"2024-09-13T17:42:06.305594+0000","last_active":"2024-09-13T17:42:07.335532+0000","last_peered":"2024-09-13T17:42:07.335532+0000","last_clean":"2024-09-13T17:42:07.335532+0000","last_became_active":"2024-09-13T17:42:06.305330+0000","last_became_peered":"2024-09-13T17:42:06.305330+0000","last_unstale":"2024-09-13T17:42:07.335532+0000","last_undegraded":"2024-09-13T17:42:07.335532+0000","last_fullsized":"2024-09-13T17:42:07.335532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:17:23.133953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.335793+0000","last_change":"2024-09-13T17:42:07.333147+0000","last_active":"2024-09-13T17:42:07.335793+0000","last_peered":"2024-09-13T17:42:07.335793+0000","last_clean":"2024-09-13T17:42:07.335793+0000","last_became_active":"2024-09-13T17:42:07.331637+0000","last_became_peered":"2024-09-13T17:42:07.331637+0000","last_unstale":"2024-09-13T17:42:07.335793+0000","last_undegraded":"2024-09-13T17:42:07.335793+0000","last_fullsized":"2024-09-13T17:42:07.335793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:25.632002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.328050+0000","last_change":"2024-09-13T17:42:06.312084+0000","last_active":"2024-09-13T17:42:07.328050+0000","last_peered":"2024-09-13T17:42:07.328050+0000","last_clean":"2024-09-13T17:42:07.328050+0000","last_became_active":"2024-09-13T17:42:06.311848+0000","last_became_peered":"2024-09-13T17:42:06.311848+0000","last_unstale":"2024-09-13T17:42:07.328050+0000","last_undegraded":"2024-09-13T17:42:07.328050+0000","last_fullsized":"2024-09-13T17:42:07.328050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:32.379434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.335857+0000","last_change":"2024-09-13T17:42:07.333648+0000","last_active":"2024-09-13T17:42:07.335857+0000","last_peered":"2024-09-13T17:42:07.335857+0000","last_clean":"2024-09-13T17:42:07.335857+0000","last_became_active":"2024-09-13T17:42:07.332563+0000","last_became_peered":"2024-09-13T17:42:07.332563+0000","last_unstale":"2024-09-13T17:42:07.335857+0000","last_undegraded":"2024-09-13T17:42:07.335857+0000","last_fullsized":"2024-09-13T17:42:07.335857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:06.934321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331832+0000","last_change":"2024-09-13T17:42:06.308815+0000","last_active":"2024-09-13T17:42:07.331832+0000","last_peered":"2024-09-13T17:42:07.331832+0000","last_clean":"2024-09-13T17:42:07.331832+0000","last_became_active":"2024-09-13T17:42:06.308407+0000","last_became_peered":"2024-09-13T17:42:06.308407+0000","last_unstale":"2024-09-13T17:42:07.331832+0000","last_undegraded":"2024-09-13T17:42:07.331832+0000","last_fullsized":"2024-09-13T17:42:07.331832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:19.237563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.331281+0000","last_change":"2024-09-13T17:42:07.315582+0000","last_active":"2024-09-13T17:42:07.331281+0000","last_peered":"2024-09-13T17:42:07.331281+0000","last_clean":"2024-09-13T17:42:07.331281+0000","last_became_active":"2024-09-13T17:42:07.312277+0000","last_became_peered":"2024-09-13T17:42:07.312277+0000","last_unstale":"2024-09-13T17:42:07.331281+0000","last_undegraded":"2024-09-13T17:42:07.331281+0000","last_fullsized":"2024-09-13T17:42:07.331281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:59:50.392102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.338321+0000","last_change":"2024-09-13T17:42:06.312228+0000","last_active":"2024-09-13T17:42:07.338321+0000","last_peered":"2024-09-13T17:42:07.338321+0000","last_clean":"2024-09-13T17:42:07.338321+0000","last_became_active":"2024-09-13T17:42:06.311930+0000","last_became_peered":"2024-09-13T17:42:06.311930+0000","last_unstale":"2024-09-13T17:42:07.338321+0000","last_undegraded":"2024-09-13T17:42:07.338321+0000","last_fullsized":"2024-09-13T17:42:07.338321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:16.993105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.330780+0000","last_change":"2024-09-13T17:42:07.330018+0000","last_active":"2024-09-13T17:42:07.330780+0000","last_peered":"2024-09-13T17:42:07.330780+0000","last_clean":"2024-09-13T17:42:07.330780+0000","last_became_active":"2024-09-13T17:42:07.329763+0000","last_became_peered":"2024-09-13T17:42:07.329763+0000","last_unstale":"2024-09-13T17:42:07.330780+0000","last_undegraded":"2024-09-13T17:42:07.330780+0000","last_fullsized":"2024-09-13T17:42:07.330780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:38:02.719445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T17:42:07.337820+0000","last_change":"2024-09-13T17:42:06.305609+0000","last_active":"2024-09-13T17:42:07.337820+0000","last_peered":"2024-09-13T17:42:07.337820+0000","last_clean":"2024-09-13T17:42:07.337820+0000","last_became_active":"2024-09-13T17:42:06.304878+0000","last_became_peered":"2024-09-13T17:42:06.304878+0000","last_unstale":"2024-09-13T17:42:07.337820+0000","last_undegraded":"2024-09-13T17:42:07.337820+0000","last_fullsized":"2024-09-13T17:42:07.337820+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:32.379434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297556,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074284,"statfs":{"total":96636764160,"available":96332066816,"internally_reserved":0,"allocated":200704,"data_stored":57550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]}]},{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297540,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074300,"statfs":{"total":96636764160,"available":96332083200,"internally_reserved":0,"allocated":184320,"data_stored":57337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]}]},{"osd":1,"up_from":9,"seq":38654705672,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298128,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073712,"statfs":{"total":96636764160,"available":96331481088,"internally_reserved":0,"allocated":782336,"data_stored":646834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]}]},{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298120,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073720,"statfs":{"total":96636764160,"available":96331489280,"internally_reserved":0,"allocated":782336,"data_stored":646869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:42:15.280 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T17:42:15.281 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-13T17:42:15.281 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-13T17:42:15.281 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-13T17:42:15.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.410+0000 7fa962a94640 1 Processor -- start 2024-09-13T17:42:15.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.410+0000 7fa962a94640 1 -- start start 2024-09-13T17:42:15.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.410+0000 7fa962a94640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa95c098ef0 0x7fa95c0992c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:15.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.410+0000 7fa962a94640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa95c067130 0x7fa95c067520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:15.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.410+0000 7fa962a94640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa95c09a0e0 con 0x7fa95c067a60 2024-09-13T17:42:15.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.410+0000 7fa962a94640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa95c097600 con 0x7fa95c067130 2024-09-13T17:42:15.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.410+0000 7fa962a94640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c096ea0 con 0x7fa95c098ef0 2024-09-13T17:42:15.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.410+0000 7fa95bfff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa95c067130 0x7fa95c067520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:15.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.410+0000 7fa960830640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa95c098ef0 0x7fa95c0992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:15.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.410+0000 7fa960830640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa95c098ef0 0x7fa95c0992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35460/0 (socket says 172.21.15.102:35460) 2024-09-13T17:42:15.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.410+0000 7fa95bfff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa95c067130 0x7fa95c067520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37462/0 (socket says 172.21.15.102:37462) 2024-09-13T17:42:15.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.410+0000 7fa961031640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fa95c067a60 0x7fa95c136750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:60894/0 (socket says 172.21.15.102:60894) 2024-09-13T17:42:15.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.410+0000 7fa960830640 1 -- 172.21.15.102:0/2028803646 learned_addr learned my addr 172.21.15.102:0/2028803646 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:15.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.411+0000 7fa960830640 1 -- 172.21.15.102:0/2028803646 >> v2:172.21.15.102:3300/0 conn(0x7fa95c067130 msgr2=0x7fa95c067520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:15.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.411+0000 7fa960830640 1 --2- 172.21.15.102:0/2028803646 >> v2:172.21.15.102:3300/0 conn(0x7fa95c067130 0x7fa95c067520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.411+0000 7fa960830640 1 -- 172.21.15.102:0/2028803646 >> v1:172.21.15.102:6789/0 conn(0x7fa95c067a60 legacy=0x7fa95c136750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:15.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.411+0000 7fa960830640 1 -- 172.21.15.102:0/2028803646 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa95c097cc0 con 0x7fa95c098ef0 2024-09-13T17:42:15.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.411+0000 7fa95bfff640 1 --2- 172.21.15.102:0/2028803646 >> v2:172.21.15.102:3300/0 conn(0x7fa95c067130 0x7fa95c067520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:15.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.411+0000 7fa960830640 1 --2- 172.21.15.102:0/2028803646 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa95c098ef0 0x7fa95c0992c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa950009910 tx=0x7fa9500091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97d90180708197f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:15.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.411+0000 7fa95b7fe640 1 -- 172.21.15.102:0/2028803646 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95002ad20 con 0x7fa95c098ef0 2024-09-13T17:42:15.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.412+0000 7fa95b7fe640 1 -- 172.21.15.102:0/2028803646 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa95002ae80 con 0x7fa95c098ef0 2024-09-13T17:42:15.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.412+0000 7fa95b7fe640 1 -- 172.21.15.102:0/2028803646 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa950032670 con 0x7fa95c098ef0 2024-09-13T17:42:15.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.412+0000 7fa962a94640 1 -- 172.21.15.102:0/2028803646 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa95c098ef0 msgr2=0x7fa95c0992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:15.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.412+0000 7fa962a94640 1 --2- 172.21.15.102:0/2028803646 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa95c098ef0 0x7fa95c0992c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa950009910 tx=0x7fa9500091d0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.412+0000 7fa962a94640 1 -- 172.21.15.102:0/2028803646 shutdown_connections 2024-09-13T17:42:15.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.412+0000 7fa962a94640 1 --2- 172.21.15.102:0/2028803646 >> v2:172.21.15.102:3300/0 conn(0x7fa95c067130 0x7fa95c067520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.412+0000 7fa962a94640 1 --2- 172.21.15.102:0/2028803646 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa95c098ef0 0x7fa95c0992c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.412+0000 7fa962a94640 1 -- 172.21.15.102:0/2028803646 >> 172.21.15.102:0/2028803646 conn(0x7fa95c05db50 msgr2=0x7fa95c05df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:15.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.412+0000 7fa962a94640 1 -- 172.21.15.102:0/2028803646 shutdown_connections 2024-09-13T17:42:15.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.412+0000 7fa962a94640 1 -- 172.21.15.102:0/2028803646 wait complete. 2024-09-13T17:42:15.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.413+0000 7fa962a94640 1 Processor -- start 2024-09-13T17:42:15.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.413+0000 7fa962a94640 1 -- start start 2024-09-13T17:42:15.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.413+0000 7fa962a94640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa95c067130 0x7fa95c120850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:15.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.413+0000 7fa962a94640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa95c067a60 0x7fa95c11cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:15.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.413+0000 7fa962a94640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa95c098ef0 0x7fa95c11d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:15.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.413+0000 7fa962a94640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c11faf0 con 0x7fa95c098ef0 2024-09-13T17:42:15.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.413+0000 7fa962a94640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c11e5c0 con 0x7fa95c067130 2024-09-13T17:42:15.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.413+0000 7fa962a94640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c0987b0 con 0x7fa95c067a60 2024-09-13T17:42:15.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.413+0000 7fa960830640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa95c067130 0x7fa95c120850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:15.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.413+0000 7fa95bfff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa95c067a60 0x7fa95c11cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:15.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.413+0000 7fa960830640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa95c067130 0x7fa95c120850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35474/0 (socket says 172.21.15.102:35474) 2024-09-13T17:42:15.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.414+0000 7fa960830640 1 -- 172.21.15.102:0/3013959254 learned_addr learned my addr 172.21.15.102:0/3013959254 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:15.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.414+0000 7fa961031640 1 --2- 172.21.15.102:0/3013959254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa95c098ef0 0x7fa95c11d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:15.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.414+0000 7fa960830640 1 -- 172.21.15.102:0/3013959254 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa95c067a60 msgr2=0x7fa95c11cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:15.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.414+0000 7fa960830640 1 --2- 172.21.15.102:0/3013959254 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa95c067a60 0x7fa95c11cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.414+0000 7fa960830640 1 -- 172.21.15.102:0/3013959254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa95c098ef0 msgr2=0x7fa95c11d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:15.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.414+0000 7fa960830640 1 --2- 172.21.15.102:0/3013959254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa95c098ef0 0x7fa95c11d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.414+0000 7fa960830640 1 -- 172.21.15.102:0/3013959254 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa948009390 con 0x7fa95c067130 2024-09-13T17:42:15.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.414+0000 7fa960830640 1 --2- 172.21.15.102:0/3013959254 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa95c067130 0x7fa95c120850 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa950009970 tx=0x7fa950032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:15.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.414+0000 7fa958ff9640 1 -- 172.21.15.102:0/3013959254 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95003b6e0 con 0x7fa95c067130 2024-09-13T17:42:15.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.414+0000 7fa958ff9640 1 -- 172.21.15.102:0/3013959254 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa95003b860 con 0x7fa95c067130 2024-09-13T17:42:15.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.414+0000 7fa962a94640 1 -- 172.21.15.102:0/3013959254 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa950009450 con 0x7fa95c067130 2024-09-13T17:42:15.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.414+0000 7fa958ff9640 1 -- 172.21.15.102:0/3013959254 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95003bc60 con 0x7fa95c067130 2024-09-13T17:42:15.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.414+0000 7fa962a94640 1 -- 172.21.15.102:0/3013959254 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa95c11da70 con 0x7fa95c067130 2024-09-13T17:42:15.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.415+0000 7fa958ff9640 1 -- 172.21.15.102:0/3013959254 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fa950043030 con 0x7fa95c067130 2024-09-13T17:42:15.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.415+0000 7fa958ff9640 1 -- 172.21.15.102:0/3013959254 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7fa950041070 con 0x7fa95c067130 2024-09-13T17:42:15.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.415+0000 7fa962a94640 1 -- 172.21.15.102:0/3013959254 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa95c11ff40 con 0x7fa95c067130 2024-09-13T17:42:15.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.419+0000 7fa958ff9640 1 -- 172.21.15.102:0/3013959254 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fa95c11ff40 con 0x7fa95c067130 2024-09-13T17:42:15.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.601+0000 7fa962a94640 1 -- 172.21.15.102:0/3013959254 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa95c11ff40 con 0x7fa95c067130 2024-09-13T17:42:15.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.601+0000 7fa958ff9640 1 -- 172.21.15.102:0/3013959254 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fa95c11ff40 con 0x7fa95c067130 2024-09-13T17:42:15.603 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:42:15.603 INFO:teuthology.orchestra.run.smithi102.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-13T17:42:15.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.603+0000 7fa962a94640 1 -- 172.21.15.102:0/3013959254 >> v1:172.21.15.102:6816/34937 conn(0x7fa940031160 legacy=0x7fa940033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:15.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.603+0000 7fa962a94640 1 -- 172.21.15.102:0/3013959254 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa95c067130 msgr2=0x7fa95c120850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:15.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.603+0000 7fa962a94640 1 --2- 172.21.15.102:0/3013959254 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa95c067130 0x7fa95c120850 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa950009970 tx=0x7fa950032000 comp rx=0 tx=0).stop 2024-09-13T17:42:15.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.603+0000 7fa962a94640 1 -- 172.21.15.102:0/3013959254 shutdown_connections 2024-09-13T17:42:15.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.603+0000 7fa962a94640 1 --2- 172.21.15.102:0/3013959254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa95c098ef0 0x7fa95c11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.603+0000 7fa962a94640 1 --2- 172.21.15.102:0/3013959254 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa95c067a60 0x7fa95c11cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.603+0000 7fa962a94640 1 --2- 172.21.15.102:0/3013959254 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa95c067130 0x7fa95c120850 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.603+0000 7fa962a94640 1 -- 172.21.15.102:0/3013959254 >> 172.21.15.102:0/3013959254 conn(0x7fa95c05db50 msgr2=0x7fa95c06ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:15.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.603+0000 7fa962a94640 1 -- 172.21.15.102:0/3013959254 shutdown_connections 2024-09-13T17:42:15.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.603+0000 7fa962a94640 1 -- 172.21.15.102:0/3013959254 wait complete. 2024-09-13T17:42:15.616 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-13T17:42:15.616 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:15.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.708+0000 7fe87db9d640 1 Processor -- start 2024-09-13T17:42:15.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.708+0000 7fe87db9d640 1 -- start start 2024-09-13T17:42:15.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.708+0000 7fe87db9d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe87806b140 0x7fe8780998c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:15.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.708+0000 7fe87db9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe87809ac20 0x7fe878099e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:15.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.708+0000 7fe87db9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe87809a340 0x7fe87812d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:15.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.708+0000 7fe87db9d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe87806a680 con 0x7fe87806b140 2024-09-13T17:42:15.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.708+0000 7fe87db9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe87806ee90 con 0x7fe87809ac20 2024-09-13T17:42:15.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.708+0000 7fe87db9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe87806ac90 con 0x7fe87809a340 2024-09-13T17:42:15.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.709+0000 7fe8777fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe87806b140 0x7fe8780998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:15.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.709+0000 7fe876ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe87809ac20 0x7fe878099e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:15.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.709+0000 7fe877fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe87809a340 0x7fe87812d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:15.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.709+0000 7fe876ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe87809ac20 0x7fe878099e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35482/0 (socket says 172.21.15.102:35482) 2024-09-13T17:42:15.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.709+0000 7fe877fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe87809a340 0x7fe87812d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:46700/0 (socket says 172.21.15.102:46700) 2024-09-13T17:42:15.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.709+0000 7fe876ffd640 1 -- 172.21.15.102:0/3220927100 learned_addr learned my addr 172.21.15.102:0/3220927100 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:15.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.709+0000 7fe877fff640 1 -- 172.21.15.102:0/3220927100 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe87809ac20 msgr2=0x7fe878099e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:15.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.709+0000 7fe877fff640 1 --2- 172.21.15.102:0/3220927100 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe87809ac20 0x7fe878099e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.709+0000 7fe877fff640 1 -- 172.21.15.102:0/3220927100 >> v2:172.21.15.102:3300/0 conn(0x7fe87806b140 msgr2=0x7fe8780998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:15.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.709+0000 7fe877fff640 1 --2- 172.21.15.102:0/3220927100 >> v2:172.21.15.102:3300/0 conn(0x7fe87806b140 0x7fe8780998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.709+0000 7fe877fff640 1 -- 172.21.15.102:0/3220927100 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe878099510 con 0x7fe87809a340 2024-09-13T17:42:15.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.709+0000 7fe876ffd640 1 --2- 172.21.15.102:0/3220927100 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe87809ac20 0x7fe878099e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:15.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.709+0000 7fe8777fe640 1 --2- 172.21.15.102:0/3220927100 >> v2:172.21.15.102:3300/0 conn(0x7fe87806b140 0x7fe8780998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:15.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.710+0000 7fe877fff640 1 --2- 172.21.15.102:0/3220927100 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe87809a340 0x7fe87812d280 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe870009e90 tx=0x7fe870009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f5c4383673988fc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:15.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.710+0000 7fe8767fc640 1 -- 172.21.15.102:0/3220927100 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe870007550 con 0x7fe87809a340 2024-09-13T17:42:15.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.710+0000 7fe8767fc640 1 -- 172.21.15.102:0/3220927100 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe87002d050 con 0x7fe87809a340 2024-09-13T17:42:15.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.710+0000 7fe8767fc640 1 -- 172.21.15.102:0/3220927100 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe870007920 con 0x7fe87809a340 2024-09-13T17:42:15.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.710+0000 7fe87db9d640 1 -- 172.21.15.102:0/3220927100 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe87809a340 msgr2=0x7fe87812d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:15.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.710+0000 7fe87db9d640 1 --2- 172.21.15.102:0/3220927100 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe87809a340 0x7fe87812d280 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe870009e90 tx=0x7fe870009780 comp rx=0 tx=0).stop 2024-09-13T17:42:15.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.710+0000 7fe87db9d640 1 -- 172.21.15.102:0/3220927100 shutdown_connections 2024-09-13T17:42:15.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.710+0000 7fe87db9d640 1 --2- 172.21.15.102:0/3220927100 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe87809a340 0x7fe87812d280 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.710+0000 7fe87db9d640 1 --2- 172.21.15.102:0/3220927100 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe87809ac20 0x7fe878099e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.710+0000 7fe87db9d640 1 --2- 172.21.15.102:0/3220927100 >> v2:172.21.15.102:3300/0 conn(0x7fe87806b140 0x7fe8780998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.710+0000 7fe87db9d640 1 -- 172.21.15.102:0/3220927100 >> 172.21.15.102:0/3220927100 conn(0x7fe87809e820 msgr2=0x7fe87809ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:15.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.710+0000 7fe87db9d640 1 -- 172.21.15.102:0/3220927100 shutdown_connections 2024-09-13T17:42:15.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.710+0000 7fe87db9d640 1 -- 172.21.15.102:0/3220927100 wait complete. 2024-09-13T17:42:15.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.711+0000 7fe87db9d640 1 Processor -- start 2024-09-13T17:42:15.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.711+0000 7fe87db9d640 1 -- start start 2024-09-13T17:42:15.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.712+0000 7fe87db9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe87806b140 0x7fe878096a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:15.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.712+0000 7fe87db9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe87809a340 0x7fe878096fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:15.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.712+0000 7fe87db9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe87809ac20 0x7fe8780974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:15.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.712+0000 7fe87db9d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe878120220 con 0x7fe87806b140 2024-09-13T17:42:15.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.712+0000 7fe87db9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe87809da90 con 0x7fe87809ac20 2024-09-13T17:42:15.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.712+0000 7fe87db9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe878069dd0 con 0x7fe87809a340 2024-09-13T17:42:15.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.712+0000 7fe8777fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe87806b140 0x7fe878096a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:15.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.712+0000 7fe8777fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe87806b140 0x7fe878096a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37476/0 (socket says 172.21.15.102:37476) 2024-09-13T17:42:15.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.712+0000 7fe8777fe640 1 -- 172.21.15.102:0/963605794 learned_addr learned my addr 172.21.15.102:0/963605794 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:15.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.712+0000 7fe876ffd640 1 --2- 172.21.15.102:0/963605794 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe87809a340 0x7fe878096fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:15.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.712+0000 7fe877fff640 1 --2- 172.21.15.102:0/963605794 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe87809ac20 0x7fe8780974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:15.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.712+0000 7fe8777fe640 1 -- 172.21.15.102:0/963605794 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe87809a340 msgr2=0x7fe878096fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:15.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.712+0000 7fe8777fe640 1 --2- 172.21.15.102:0/963605794 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe87809a340 0x7fe878096fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.712+0000 7fe8777fe640 1 -- 172.21.15.102:0/963605794 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe87809ac20 msgr2=0x7fe8780974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:15.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.712+0000 7fe8777fe640 1 --2- 172.21.15.102:0/963605794 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe87809ac20 0x7fe8780974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.712+0000 7fe8777fe640 1 -- 172.21.15.102:0/963605794 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe868009390 con 0x7fe87806b140 2024-09-13T17:42:15.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.712+0000 7fe876ffd640 1 --2- 172.21.15.102:0/963605794 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe87809a340 0x7fe878096fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:15.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.712+0000 7fe8777fe640 1 --2- 172.21.15.102:0/963605794 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe87806b140 0x7fe878096a70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe86800ea70 tx=0x7fe8680090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:15.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.712+0000 7fe85f7fe640 1 -- 172.21.15.102:0/963605794 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86800ca10 con 0x7fe87806b140 2024-09-13T17:42:15.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.712+0000 7fe85f7fe640 1 -- 172.21.15.102:0/963605794 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86800cb70 con 0x7fe87806b140 2024-09-13T17:42:15.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.712+0000 7fe85f7fe640 1 -- 172.21.15.102:0/963605794 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe868013470 con 0x7fe87806b140 2024-09-13T17:42:15.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.712+0000 7fe87db9d640 1 -- 172.21.15.102:0/963605794 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe86c009450 con 0x7fe87806b140 2024-09-13T17:42:15.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.712+0000 7fe87db9d640 1 -- 172.21.15.102:0/963605794 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe870009a20 con 0x7fe87806b140 2024-09-13T17:42:15.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.713+0000 7fe85f7fe640 1 -- 172.21.15.102:0/963605794 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fe868013820 con 0x7fe87806b140 2024-09-13T17:42:15.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.713+0000 7fe87db9d640 1 -- 172.21.15.102:0/963605794 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe878121380 con 0x7fe87806b140 2024-09-13T17:42:15.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.713+0000 7fe85f7fe640 1 -- 172.21.15.102:0/963605794 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7fe8680423e0 con 0x7fe87806b140 2024-09-13T17:42:15.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.718+0000 7fe85f7fe640 1 -- 172.21.15.102:0/963605794 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fe868021020 con 0x7fe87806b140 2024-09-13T17:42:15.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.872+0000 7fe87db9d640 1 -- 172.21.15.102:0/963605794 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe878064000 con 0x7fe87806b140 2024-09-13T17:42:15.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.872+0000 7fe85f7fe640 1 -- 172.21.15.102:0/963605794 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+11375 (secure 0 0 0) 0x7fe868023ad0 con 0x7fe87806b140 2024-09-13T17:42:15.874 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:42:15.875 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":20,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:42:07.375548+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:04.672265+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:05.583134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:15.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.874+0000 7fe87db9d640 1 -- 172.21.15.102:0/963605794 >> v1:172.21.15.102:6816/34937 conn(0x7fe864031160 legacy=0x7fe864033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:15.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.874+0000 7fe87db9d640 1 -- 172.21.15.102:0/963605794 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe87806b140 msgr2=0x7fe878096a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:15.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.874+0000 7fe87db9d640 1 --2- 172.21.15.102:0/963605794 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe87806b140 0x7fe878096a70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe86800ea70 tx=0x7fe8680090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.877 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.874+0000 7fe87db9d640 1 -- 172.21.15.102:0/963605794 shutdown_connections 2024-09-13T17:42:15.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.874+0000 7fe87db9d640 1 --2- 172.21.15.102:0/963605794 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe87809ac20 0x7fe8780974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.874+0000 7fe87db9d640 1 --2- 172.21.15.102:0/963605794 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe87809a340 0x7fe878096fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.874+0000 7fe87db9d640 1 --2- 172.21.15.102:0/963605794 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe87806b140 0x7fe878096a70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.874+0000 7fe87db9d640 1 -- 172.21.15.102:0/963605794 >> 172.21.15.102:0/963605794 conn(0x7fe87809e820 msgr2=0x7fe87806e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:15.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.874+0000 7fe87db9d640 1 -- 172.21.15.102:0/963605794 shutdown_connections 2024-09-13T17:42:15.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.874+0000 7fe87db9d640 1 -- 172.21.15.102:0/963605794 wait complete. 2024-09-13T17:42:15.887 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:54.442612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T17:41:59.076782+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 16, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:42:04.672265+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:42:05.583134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:42:15.888 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:42:15.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.981+0000 7f6fa44c9640 1 Processor -- start 2024-09-13T17:42:15.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.981+0000 7f6fa44c9640 1 -- start start 2024-09-13T17:42:15.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.982+0000 7f6fa44c9640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6f9c067130 0x7f6f9c12d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:15.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.982+0000 7f6fa44c9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6f9c12d8d0 0x7f6f9c12dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:15.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.982+0000 7f6fa44c9640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f9c067e80 con 0x7f6f9c098c80 2024-09-13T17:42:15.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.982+0000 7f6fa44c9640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f9c067500 con 0x7f6f9c067130 2024-09-13T17:42:15.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.982+0000 7f6fa44c9640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c099ea0 con 0x7f6f9c12d8d0 2024-09-13T17:42:15.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.982+0000 7f6fa2265640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6f9c067130 0x7f6f9c12d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:15.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.982+0000 7f6fa1a64640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6f9c12d8d0 0x7f6f9c12dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:15.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.982+0000 7f6fa2a66640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f6f9c098c80 0x7f6f9c0990e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:60910/0 (socket says 172.21.15.102:60910) 2024-09-13T17:42:15.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.982+0000 7f6fa2a66640 1 -- 172.21.15.102:0/2367427978 learned_addr learned my addr 172.21.15.102:0/2367427978 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:42:15.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.982+0000 7f6fa1a64640 1 -- 172.21.15.102:0/2367427978 >> v2:172.21.15.102:3300/0 conn(0x7f6f9c067130 msgr2=0x7f6f9c12d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:15.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.982+0000 7f6fa1a64640 1 --2- 172.21.15.102:0/2367427978 >> v2:172.21.15.102:3300/0 conn(0x7f6f9c067130 0x7f6f9c12d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.982+0000 7f6fa1a64640 1 -- 172.21.15.102:0/2367427978 >> v1:172.21.15.102:6789/0 conn(0x7f6f9c098c80 legacy=0x7f6f9c0990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:15.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.982+0000 7f6fa1a64640 1 -- 172.21.15.102:0/2367427978 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9c0680b0 con 0x7f6f9c12d8d0 2024-09-13T17:42:15.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.982+0000 7f6fa1263640 1 -- 172.21.15.102:0/2367427978 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1387486304 0 0) 0x7f6f9c067e80 con 0x7f6f9c098c80 2024-09-13T17:42:15.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.982+0000 7f6fa2265640 1 --2- 172.21.15.102:0/2367427978 >> v2:172.21.15.102:3300/0 conn(0x7f6f9c067130 0x7f6f9c12d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:15.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.983+0000 7f6fa1a64640 1 --2- 172.21.15.102:0/2367427978 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6f9c12d8d0 0x7f6f9c12dce0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009850 tx=0x7f6f8c0090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e97f75db382ae3f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:15.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.983+0000 7f6fa1263640 1 -- 172.21.15.102:0/2367427978 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c02ad80 con 0x7f6f9c12d8d0 2024-09-13T17:42:15.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.983+0000 7f6fa1263640 1 -- 172.21.15.102:0/2367427978 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f8c032440 con 0x7f6f9c12d8d0 2024-09-13T17:42:15.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.983+0000 7f6fa1263640 1 -- 172.21.15.102:0/2367427978 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c0327e0 con 0x7f6f9c12d8d0 2024-09-13T17:42:15.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.983+0000 7f6fa44c9640 1 -- 172.21.15.102:0/2367427978 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6f9c12d8d0 msgr2=0x7f6f9c12dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:15.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.983+0000 7f6fa44c9640 1 --2- 172.21.15.102:0/2367427978 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6f9c12d8d0 0x7f6f9c12dce0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009850 tx=0x7f6f8c0090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.984+0000 7f6fa44c9640 1 -- 172.21.15.102:0/2367427978 shutdown_connections 2024-09-13T17:42:15.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.984+0000 7f6fa44c9640 1 --2- 172.21.15.102:0/2367427978 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6f9c12d8d0 0x7f6f9c12dce0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.984+0000 7f6fa44c9640 1 --2- 172.21.15.102:0/2367427978 >> v2:172.21.15.102:3300/0 conn(0x7f6f9c067130 0x7f6f9c12d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.984+0000 7f6fa44c9640 1 -- 172.21.15.102:0/2367427978 >> 172.21.15.102:0/2367427978 conn(0x7f6f9c0a0420 msgr2=0x7f6f9c05db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:15.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.984+0000 7f6fa44c9640 1 -- 172.21.15.102:0/2367427978 shutdown_connections 2024-09-13T17:42:15.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.984+0000 7f6fa44c9640 1 -- 172.21.15.102:0/2367427978 wait complete. 2024-09-13T17:42:15.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.984+0000 7f6fa44c9640 1 Processor -- start 2024-09-13T17:42:15.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.984+0000 7f6fa44c9640 1 -- start start 2024-09-13T17:42:15.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.985+0000 7f6fa44c9640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6f9c067130 0x7f6f9c118ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:15.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.985+0000 7f6fa44c9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6f9c098c80 0x7f6f9c1191e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:15.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.985+0000 7f6fa44c9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f9c12d8d0 0x7f6f9c13c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:15.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.985+0000 7f6fa44c9640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c11aaa0 con 0x7f6f9c12d8d0 2024-09-13T17:42:15.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.985+0000 7f6fa44c9640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c09d8d0 con 0x7f6f9c067130 2024-09-13T17:42:15.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.985+0000 7f6fa44c9640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c11b970 con 0x7f6f9c098c80 2024-09-13T17:42:15.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.985+0000 7f6fa1a64640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6f9c098c80 0x7f6f9c1191e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:15.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.985+0000 7f6fa1a64640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6f9c098c80 0x7f6f9c1191e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:46734/0 (socket says 172.21.15.102:46734) 2024-09-13T17:42:15.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.985+0000 7f6fa1a64640 1 -- 172.21.15.102:0/1720110314 learned_addr learned my addr 172.21.15.102:0/1720110314 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:15.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.985+0000 7f6fa2265640 1 --2- 172.21.15.102:0/1720110314 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6f9c067130 0x7f6f9c118ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:15.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.985+0000 7f6fa2a66640 1 --2- 172.21.15.102:0/1720110314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f9c12d8d0 0x7f6f9c13c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:15.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.985+0000 7f6fa1a64640 1 -- 172.21.15.102:0/1720110314 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6f9c067130 msgr2=0x7f6f9c118ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:15.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.985+0000 7f6fa1a64640 1 --2- 172.21.15.102:0/1720110314 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6f9c067130 0x7f6f9c118ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.985+0000 7f6fa1a64640 1 -- 172.21.15.102:0/1720110314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f9c12d8d0 msgr2=0x7f6f9c13c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:15.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.985+0000 7f6fa1a64640 1 --2- 172.21.15.102:0/1720110314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f9c12d8d0 0x7f6f9c13c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.985+0000 7f6fa1a64640 1 -- 172.21.15.102:0/1720110314 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f98009390 con 0x7f6f9c098c80 2024-09-13T17:42:15.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.985+0000 7f6fa2a66640 1 --2- 172.21.15.102:0/1720110314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f9c12d8d0 0x7f6f9c13c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:15.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.985+0000 7f6fa1a64640 1 --2- 172.21.15.102:0/1720110314 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6f9c098c80 0x7f6f9c1191e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c02a6b0 tx=0x7f6f8c032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:15.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.986+0000 7f6f927fc640 1 -- 172.21.15.102:0/1720110314 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c031de0 con 0x7f6f9c098c80 2024-09-13T17:42:15.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.986+0000 7f6f927fc640 1 -- 172.21.15.102:0/1720110314 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f8c03d420 con 0x7f6f9c098c80 2024-09-13T17:42:15.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.986+0000 7f6fa44c9640 1 -- 172.21.15.102:0/1720110314 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f8c009390 con 0x7f6f9c098c80 2024-09-13T17:42:15.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.986+0000 7f6f927fc640 1 -- 172.21.15.102:0/1720110314 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c03d7c0 con 0x7f6f9c098c80 2024-09-13T17:42:15.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.986+0000 7f6fa44c9640 1 -- 172.21.15.102:0/1720110314 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f9c11c660 con 0x7f6f9c098c80 2024-09-13T17:42:15.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.986+0000 7f6f927fc640 1 -- 172.21.15.102:0/1720110314 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f6f8c039040 con 0x7f6f9c098c80 2024-09-13T17:42:15.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.987+0000 7f6f927fc640 1 -- 172.21.15.102:0/1720110314 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7f6f8c062860 con 0x7f6f9c098c80 2024-09-13T17:42:15.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.987+0000 7f6fa44c9640 1 -- 172.21.15.102:0/1720110314 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f9c11bb90 con 0x7f6f9c098c80 2024-09-13T17:42:15.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:15.991+0000 7f6f927fc640 1 -- 172.21.15.102:0/1720110314 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f6f8c037080 con 0x7f6f9c098c80 2024-09-13T17:42:16.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.141+0000 7f6fa44c9640 1 -- 172.21.15.102:0/1720110314 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6f9c064000 con 0x7f6f9c098c80 2024-09-13T17:42:16.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.141+0000 7f6f927fc640 1 -- 172.21.15.102:0/1720110314 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6f8c042030 con 0x7f6f9c098c80 2024-09-13T17:42:16.143 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-09-13T17:42:16.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.143+0000 7f6fa44c9640 1 -- 172.21.15.102:0/1720110314 >> v1:172.21.15.102:6816/34937 conn(0x7f6f80031160 legacy=0x7f6f80033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:16.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.143+0000 7f6fa44c9640 1 -- 172.21.15.102:0/1720110314 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6f9c098c80 msgr2=0x7f6f9c1191e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:16.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.143+0000 7f6fa44c9640 1 --2- 172.21.15.102:0/1720110314 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6f9c098c80 0x7f6f9c1191e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c02a6b0 tx=0x7f6f8c032000 comp rx=0 tx=0).stop 2024-09-13T17:42:16.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.143+0000 7f6fa44c9640 1 -- 172.21.15.102:0/1720110314 shutdown_connections 2024-09-13T17:42:16.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.143+0000 7f6fa44c9640 1 --2- 172.21.15.102:0/1720110314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f9c12d8d0 0x7f6f9c13c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.143+0000 7f6fa44c9640 1 --2- 172.21.15.102:0/1720110314 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6f9c098c80 0x7f6f9c1191e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.143+0000 7f6fa44c9640 1 --2- 172.21.15.102:0/1720110314 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6f9c067130 0x7f6f9c118ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.143+0000 7f6fa44c9640 1 -- 172.21.15.102:0/1720110314 >> 172.21.15.102:0/1720110314 conn(0x7f6f9c0a0420 msgr2=0x7f6f9c06dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:16.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.144+0000 7f6fa44c9640 1 -- 172.21.15.102:0/1720110314 shutdown_connections 2024-09-13T17:42:16.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.144+0000 7f6fa44c9640 1 -- 172.21.15.102:0/1720110314 wait complete. 2024-09-13T17:42:16.156 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-13T17:42:16.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.250+0000 7f298c79d640 1 Processor -- start 2024-09-13T17:42:16.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.250+0000 7f298c79d640 1 -- start start 2024-09-13T17:42:16.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.251+0000 7f298c79d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f298409c510 0x7f298409a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:16.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.251+0000 7f298c79d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f298409b250 0x7f29841377f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:16.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.251+0000 7f298c79d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f298406f740 con 0x7f298409ae50 2024-09-13T17:42:16.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.251+0000 7f298c79d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2984095f90 con 0x7f298409c510 2024-09-13T17:42:16.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.251+0000 7f298c79d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f298409c8e0 con 0x7f298409b250 2024-09-13T17:42:16.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.251+0000 7f298a539640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f298409c510 0x7f298409a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:16.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.251+0000 7f2989d38640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f298409ae50 0x7f29841340c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:60924/0 (socket says 172.21.15.102:60924) 2024-09-13T17:42:16.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.251+0000 7f298ad3a640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f298409b250 0x7f29841377f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:16.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.251+0000 7f2989d38640 1 -- 172.21.15.102:0/3143303458 learned_addr learned my addr 172.21.15.102:0/3143303458 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:42:16.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.251+0000 7f298ad3a640 1 -- 172.21.15.102:0/3143303458 >> v2:172.21.15.102:3300/0 conn(0x7f298409c510 msgr2=0x7f298409a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:16.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.251+0000 7f298ad3a640 1 --2- 172.21.15.102:0/3143303458 >> v2:172.21.15.102:3300/0 conn(0x7f298409c510 0x7f298409a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.251+0000 7f298ad3a640 1 -- 172.21.15.102:0/3143303458 >> v1:172.21.15.102:6789/0 conn(0x7f298409ae50 legacy=0x7f29841340c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:16.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.251+0000 7f298ad3a640 1 -- 172.21.15.102:0/3143303458 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2984096b00 con 0x7f298409b250 2024-09-13T17:42:16.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.251+0000 7f298a539640 1 --2- 172.21.15.102:0/3143303458 >> v2:172.21.15.102:3300/0 conn(0x7f298409c510 0x7f298409a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:16.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.252+0000 7f298ad3a640 1 --2- 172.21.15.102:0/3143303458 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f298409b250 0x7f29841377f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f297c009e60 tx=0x7f297c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d4b992247375b96 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:16.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.252+0000 7f2989537640 1 -- 172.21.15.102:0/3143303458 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c007550 con 0x7f298409b250 2024-09-13T17:42:16.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.252+0000 7f2989537640 1 -- 172.21.15.102:0/3143303458 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297c02d050 con 0x7f298409b250 2024-09-13T17:42:16.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.252+0000 7f2989537640 1 -- 172.21.15.102:0/3143303458 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c0078e0 con 0x7f298409b250 2024-09-13T17:42:16.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.252+0000 7f298c79d640 1 -- 172.21.15.102:0/3143303458 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f298409b250 msgr2=0x7f29841377f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:16.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.252+0000 7f298c79d640 1 --2- 172.21.15.102:0/3143303458 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f298409b250 0x7f29841377f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f297c009e60 tx=0x7f297c009780 comp rx=0 tx=0).stop 2024-09-13T17:42:16.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.252+0000 7f298c79d640 1 -- 172.21.15.102:0/3143303458 shutdown_connections 2024-09-13T17:42:16.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.252+0000 7f298c79d640 1 --2- 172.21.15.102:0/3143303458 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f298409b250 0x7f29841377f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.252+0000 7f298c79d640 1 --2- 172.21.15.102:0/3143303458 >> v2:172.21.15.102:3300/0 conn(0x7f298409c510 0x7f298409a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.252+0000 7f298c79d640 1 -- 172.21.15.102:0/3143303458 >> 172.21.15.102:0/3143303458 conn(0x7f298409f960 msgr2=0x7f298409ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:16.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.253+0000 7f298c79d640 1 -- 172.21.15.102:0/3143303458 shutdown_connections 2024-09-13T17:42:16.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.253+0000 7f298c79d640 1 -- 172.21.15.102:0/3143303458 wait complete. 2024-09-13T17:42:16.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.253+0000 7f298c79d640 1 Processor -- start 2024-09-13T17:42:16.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.253+0000 7f298c79d640 1 -- start start 2024-09-13T17:42:16.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.254+0000 7f298c79d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f298409ae50 0x7f2984094020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:16.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.254+0000 7f298c79d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f298409b250 0x7f29840900e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:16.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.254+0000 7f298c79d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f298409c510 0x7f2984090620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:16.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.254+0000 7f298c79d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2984092150 con 0x7f298409ae50 2024-09-13T17:42:16.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.254+0000 7f298c79d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2984095340 con 0x7f298409c510 2024-09-13T17:42:16.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.254+0000 7f298c79d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2984091550 con 0x7f298409b250 2024-09-13T17:42:16.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.254+0000 7f298a539640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f298409ae50 0x7f2984094020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:16.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.254+0000 7f298a539640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f298409ae50 0x7f2984094020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37514/0 (socket says 172.21.15.102:37514) 2024-09-13T17:42:16.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.254+0000 7f298ad3a640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f298409c510 0x7f2984090620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:16.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.254+0000 7f298a539640 1 -- 172.21.15.102:0/201883912 learned_addr learned my addr 172.21.15.102:0/201883912 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:16.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.254+0000 7f2989d38640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f298409b250 0x7f29840900e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:16.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.254+0000 7f298a539640 1 -- 172.21.15.102:0/201883912 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f298409b250 msgr2=0x7f29840900e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:16.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.254+0000 7f298a539640 1 --2- 172.21.15.102:0/201883912 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f298409b250 0x7f29840900e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.254+0000 7f298a539640 1 -- 172.21.15.102:0/201883912 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f298409c510 msgr2=0x7f2984090620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:16.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.254+0000 7f298a539640 1 --2- 172.21.15.102:0/201883912 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f298409c510 0x7f2984090620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.254+0000 7f298a539640 1 -- 172.21.15.102:0/201883912 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2980009390 con 0x7f298409ae50 2024-09-13T17:42:16.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.254+0000 7f298a539640 1 --2- 172.21.15.102:0/201883912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f298409ae50 0x7f2984094020 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f298000ecb0 tx=0x7f29800090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:16.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.254+0000 7f297a7fc640 1 -- 172.21.15.102:0/201883912 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298000cbc0 con 0x7f298409ae50 2024-09-13T17:42:16.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.254+0000 7f297a7fc640 1 -- 172.21.15.102:0/201883912 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f298000cd20 con 0x7f298409ae50 2024-09-13T17:42:16.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.254+0000 7f298c79d640 1 -- 172.21.15.102:0/201883912 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f297c009a20 con 0x7f298409ae50 2024-09-13T17:42:16.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.254+0000 7f297a7fc640 1 -- 172.21.15.102:0/201883912 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2980013680 con 0x7f298409ae50 2024-09-13T17:42:16.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.254+0000 7f298c79d640 1 -- 172.21.15.102:0/201883912 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2984093840 con 0x7f298409ae50 2024-09-13T17:42:16.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.255+0000 7f297a7fc640 1 -- 172.21.15.102:0/201883912 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f29800139f0 con 0x7f298409ae50 2024-09-13T17:42:16.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.255+0000 7f297a7fc640 1 -- 172.21.15.102:0/201883912 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7f2980042440 con 0x7f298409ae50 2024-09-13T17:42:16.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.255+0000 7f298c79d640 1 -- 172.21.15.102:0/201883912 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29840935e0 con 0x7f298409ae50 2024-09-13T17:42:16.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.260+0000 7f297a7fc640 1 -- 172.21.15.102:0/201883912 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f29840935e0 con 0x7f298409ae50 2024-09-13T17:42:16.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.413+0000 7f298c79d640 1 -- 172.21.15.102:0/201883912 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f29840935e0 con 0x7f298409ae50 2024-09-13T17:42:16.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.413+0000 7f297a7fc640 1 -- 172.21.15.102:0/201883912 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f29840935e0 con 0x7f298409ae50 2024-09-13T17:42:16.415 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-09-13T17:42:16.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.415+0000 7f298c79d640 1 -- 172.21.15.102:0/201883912 >> v1:172.21.15.102:6816/34937 conn(0x7f2968031190 legacy=0x7f2968033640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:16.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.415+0000 7f298c79d640 1 -- 172.21.15.102:0/201883912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f298409ae50 msgr2=0x7f2984094020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:16.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.415+0000 7f298c79d640 1 --2- 172.21.15.102:0/201883912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f298409ae50 0x7f2984094020 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f298000ecb0 tx=0x7f29800090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.415+0000 7f298c79d640 1 -- 172.21.15.102:0/201883912 shutdown_connections 2024-09-13T17:42:16.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.415+0000 7f298c79d640 1 --2- 172.21.15.102:0/201883912 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f298409c510 0x7f2984090620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.415+0000 7f298c79d640 1 --2- 172.21.15.102:0/201883912 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f298409b250 0x7f29840900e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.415+0000 7f298c79d640 1 --2- 172.21.15.102:0/201883912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f298409ae50 0x7f2984094020 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.415+0000 7f298c79d640 1 -- 172.21.15.102:0/201883912 >> 172.21.15.102:0/201883912 conn(0x7f298409f960 msgr2=0x7f298409d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:16.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.415+0000 7f298c79d640 1 -- 172.21.15.102:0/201883912 shutdown_connections 2024-09-13T17:42:16.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.415+0000 7f298c79d640 1 -- 172.21.15.102:0/201883912 wait complete. 2024-09-13T17:42:16.427 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:42:16.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.521+0000 7fd1c875d640 1 Processor -- start 2024-09-13T17:42:16.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.521+0000 7fd1c875d640 1 -- start start 2024-09-13T17:42:16.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.522+0000 7fd1c875d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd1c0096a70 0x7fd1c012c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:16.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.522+0000 7fd1c875d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd1c012c900 0x7fd1c0098410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:16.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.522+0000 7fd1c875d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd1c012ccd0 0x7fd1c0098950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:16.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.522+0000 7fd1c875d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1c0096e40 con 0x7fd1c0096a70 2024-09-13T17:42:16.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.522+0000 7fd1c875d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c0099540 con 0x7fd1c012c900 2024-09-13T17:42:16.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.522+0000 7fd1c875d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c009a8e0 con 0x7fd1c012ccd0 2024-09-13T17:42:16.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.522+0000 7fd1c6cfa640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd1c012ccd0 0x7fd1c0098950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:16.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.522+0000 7fd1c64f9640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd1c0096a70 0x7fd1c012c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:16.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.522+0000 7fd1c6cfa640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd1c012ccd0 0x7fd1c0098950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:46776/0 (socket says 172.21.15.102:46776) 2024-09-13T17:42:16.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.522+0000 7fd1c64f9640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd1c0096a70 0x7fd1c012c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37516/0 (socket says 172.21.15.102:37516) 2024-09-13T17:42:16.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.522+0000 7fd1c6cfa640 1 -- 172.21.15.102:0/3061663638 learned_addr learned my addr 172.21.15.102:0/3061663638 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:16.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.522+0000 7fd1c5cf8640 1 --2- 172.21.15.102:0/3061663638 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd1c012c900 0x7fd1c0098410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:16.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.522+0000 7fd1c6cfa640 1 -- 172.21.15.102:0/3061663638 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd1c012c900 msgr2=0x7fd1c0098410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:16.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.522+0000 7fd1c6cfa640 1 --2- 172.21.15.102:0/3061663638 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd1c012c900 0x7fd1c0098410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.522+0000 7fd1c6cfa640 1 -- 172.21.15.102:0/3061663638 >> v2:172.21.15.102:3300/0 conn(0x7fd1c0096a70 msgr2=0x7fd1c012c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:16.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.522+0000 7fd1c6cfa640 1 --2- 172.21.15.102:0/3061663638 >> v2:172.21.15.102:3300/0 conn(0x7fd1c0096a70 0x7fd1c012c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.522+0000 7fd1c6cfa640 1 -- 172.21.15.102:0/3061663638 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c0099760 con 0x7fd1c012ccd0 2024-09-13T17:42:16.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.522+0000 7fd1c5cf8640 1 --2- 172.21.15.102:0/3061663638 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd1c012c900 0x7fd1c0098410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:16.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.523+0000 7fd1c6cfa640 1 --2- 172.21.15.102:0/3061663638 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd1c012ccd0 0x7fd1c0098950 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8009f50 tx=0x7fd1b8009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3260eed2aa3b7752 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:16.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.523+0000 7fd1c54f7640 1 -- 172.21.15.102:0/3061663638 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8031400 con 0x7fd1c012ccd0 2024-09-13T17:42:16.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.523+0000 7fd1c54f7640 1 -- 172.21.15.102:0/3061663638 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b802d050 con 0x7fd1c012ccd0 2024-09-13T17:42:16.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.523+0000 7fd1c54f7640 1 -- 172.21.15.102:0/3061663638 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b80317d0 con 0x7fd1c012ccd0 2024-09-13T17:42:16.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.524+0000 7fd1c875d640 1 -- 172.21.15.102:0/3061663638 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd1c012ccd0 msgr2=0x7fd1c0098950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:16.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.524+0000 7fd1c875d640 1 --2- 172.21.15.102:0/3061663638 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd1c012ccd0 0x7fd1c0098950 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8009f50 tx=0x7fd1b8009840 comp rx=0 tx=0).stop 2024-09-13T17:42:16.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.524+0000 7fd1c875d640 1 -- 172.21.15.102:0/3061663638 shutdown_connections 2024-09-13T17:42:16.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.524+0000 7fd1c875d640 1 --2- 172.21.15.102:0/3061663638 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd1c012ccd0 0x7fd1c0098950 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.524+0000 7fd1c875d640 1 --2- 172.21.15.102:0/3061663638 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd1c012c900 0x7fd1c0098410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.524+0000 7fd1c875d640 1 --2- 172.21.15.102:0/3061663638 >> v2:172.21.15.102:3300/0 conn(0x7fd1c0096a70 0x7fd1c012c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.524+0000 7fd1c875d640 1 -- 172.21.15.102:0/3061663638 >> 172.21.15.102:0/3061663638 conn(0x7fd1c009e6b0 msgr2=0x7fd1c009eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:16.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.524+0000 7fd1c875d640 1 -- 172.21.15.102:0/3061663638 shutdown_connections 2024-09-13T17:42:16.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.524+0000 7fd1c875d640 1 -- 172.21.15.102:0/3061663638 wait complete. 2024-09-13T17:42:16.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.524+0000 7fd1c875d640 1 Processor -- start 2024-09-13T17:42:16.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.524+0000 7fd1c875d640 1 -- start start 2024-09-13T17:42:16.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.525+0000 7fd1c875d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1c0096a70 0x7fd1c013a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:16.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.525+0000 7fd1c875d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd1c012c900 0x7fd1c013ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:16.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.525+0000 7fd1c875d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd1c012ccd0 0x7fd1c01150d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:16.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.525+0000 7fd1c875d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c0116b20 con 0x7fd1c0096a70 2024-09-13T17:42:16.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.525+0000 7fd1c875d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c012f3f0 con 0x7fd1c012ccd0 2024-09-13T17:42:16.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.525+0000 7fd1c875d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c0118490 con 0x7fd1c012c900 2024-09-13T17:42:16.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.525+0000 7fd1c64f9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1c0096a70 0x7fd1c013a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:16.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.525+0000 7fd1c5cf8640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd1c012c900 0x7fd1c013ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:16.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.525+0000 7fd1c6cfa640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd1c012ccd0 0x7fd1c01150d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:16.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.525+0000 7fd1c64f9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1c0096a70 0x7fd1c013a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37518/0 (socket says 172.21.15.102:37518) 2024-09-13T17:42:16.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.525+0000 7fd1c5cf8640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd1c012c900 0x7fd1c013ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:46790/0 (socket says 172.21.15.102:46790) 2024-09-13T17:42:16.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.525+0000 7fd1c64f9640 1 -- 172.21.15.102:0/809583322 learned_addr learned my addr 172.21.15.102:0/809583322 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:16.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.525+0000 7fd1c64f9640 1 -- 172.21.15.102:0/809583322 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd1c012c900 msgr2=0x7fd1c013ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:16.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.525+0000 7fd1c64f9640 1 --2- 172.21.15.102:0/809583322 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd1c012c900 0x7fd1c013ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.525+0000 7fd1c64f9640 1 -- 172.21.15.102:0/809583322 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd1c012ccd0 msgr2=0x7fd1c01150d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:16.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.525+0000 7fd1c64f9640 1 --2- 172.21.15.102:0/809583322 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd1c012ccd0 0x7fd1c01150d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.525+0000 7fd1c64f9640 1 -- 172.21.15.102:0/809583322 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b0009390 con 0x7fd1c0096a70 2024-09-13T17:42:16.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.525+0000 7fd1c6cfa640 1 --2- 172.21.15.102:0/809583322 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd1c012ccd0 0x7fd1c01150d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:16.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.525+0000 7fd1c64f9640 1 --2- 172.21.15.102:0/809583322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1c0096a70 0x7fd1c013a630 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00ddc0 tx=0x7fd1bc00f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:16.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.526+0000 7fd1b67fc640 1 -- 172.21.15.102:0/809583322 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc00d2e0 con 0x7fd1c0096a70 2024-09-13T17:42:16.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.526+0000 7fd1b67fc640 1 -- 172.21.15.102:0/809583322 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1bc013040 con 0x7fd1c0096a70 2024-09-13T17:42:16.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.526+0000 7fd1c875d640 1 -- 172.21.15.102:0/809583322 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1b8009ae0 con 0x7fd1c0096a70 2024-09-13T17:42:16.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.526+0000 7fd1b67fc640 1 -- 172.21.15.102:0/809583322 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc010d60 con 0x7fd1c0096a70 2024-09-13T17:42:16.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.526+0000 7fd1c875d640 1 -- 172.21.15.102:0/809583322 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c009a030 con 0x7fd1c0096a70 2024-09-13T17:42:16.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.526+0000 7fd1b67fc640 1 -- 172.21.15.102:0/809583322 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fd1bc018690 con 0x7fd1c0096a70 2024-09-13T17:42:16.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.526+0000 7fd1c875d640 1 -- 172.21.15.102:0/809583322 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c0129c90 con 0x7fd1c0096a70 2024-09-13T17:42:16.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.527+0000 7fd1b67fc640 1 -- 172.21.15.102:0/809583322 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7fd1bc046180 con 0x7fd1c0096a70 2024-09-13T17:42:16.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.531+0000 7fd1b67fc640 1 -- 172.21.15.102:0/809583322 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fd1c0129c90 con 0x7fd1c0096a70 2024-09-13T17:42:16.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.684+0000 7fd1c875d640 1 -- 172.21.15.102:0/809583322 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd1c0129c90 con 0x7fd1c0096a70 2024-09-13T17:42:16.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.684+0000 7fd1b67fc640 1 -- 172.21.15.102:0/809583322 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd1c0129c90 con 0x7fd1c0096a70 2024-09-13T17:42:16.686 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-09-13T17:42:16.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.686+0000 7fd1c875d640 1 -- 172.21.15.102:0/809583322 >> v1:172.21.15.102:6816/34937 conn(0x7fd1a4031190 legacy=0x7fd1a4033640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:16.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.686+0000 7fd1c875d640 1 -- 172.21.15.102:0/809583322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1c0096a70 msgr2=0x7fd1c013a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:16.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.686+0000 7fd1c875d640 1 --2- 172.21.15.102:0/809583322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1c0096a70 0x7fd1c013a630 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00ddc0 tx=0x7fd1bc00f040 comp rx=0 tx=0).stop 2024-09-13T17:42:16.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.686+0000 7fd1c875d640 1 -- 172.21.15.102:0/809583322 shutdown_connections 2024-09-13T17:42:16.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.686+0000 7fd1c875d640 1 --2- 172.21.15.102:0/809583322 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd1c012ccd0 0x7fd1c01150d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.686+0000 7fd1c875d640 1 --2- 172.21.15.102:0/809583322 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd1c012c900 0x7fd1c013ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.686+0000 7fd1c875d640 1 --2- 172.21.15.102:0/809583322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1c0096a70 0x7fd1c013a630 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.686+0000 7fd1c875d640 1 -- 172.21.15.102:0/809583322 >> 172.21.15.102:0/809583322 conn(0x7fd1c009e6b0 msgr2=0x7fd1c0129f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:16.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.687+0000 7fd1c875d640 1 -- 172.21.15.102:0/809583322 shutdown_connections 2024-09-13T17:42:16.689 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.687+0000 7fd1c875d640 1 -- 172.21.15.102:0/809583322 wait complete. 2024-09-13T17:42:16.699 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:42:16.804 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.794+0000 7f4a7e15d640 1 Processor -- start 2024-09-13T17:42:16.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.794+0000 7f4a7e15d640 1 -- start start 2024-09-13T17:42:16.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.794+0000 7f4a7e15d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4a780689d0 0x7f4a7809d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:16.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.794+0000 7f4a7e15d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a7809e190 0x7f4a7812fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:16.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.794+0000 7f4a7e15d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a78096b10 con 0x7f4a7809dd70 2024-09-13T17:42:16.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.794+0000 7f4a7e15d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a78068da0 con 0x7f4a780689d0 2024-09-13T17:42:16.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.794+0000 7f4a7e15d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a78096db0 con 0x7f4a7809e190 2024-09-13T17:42:16.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.794+0000 7f4a77fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a7809e190 0x7f4a7812fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:16.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.794+0000 7f4a777fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4a780689d0 0x7f4a7809d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:16.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.794+0000 7f4a76ffd640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f4a7809dd70 0x7f4a7812f610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:60934/0 (socket says 172.21.15.102:60934) 2024-09-13T17:42:16.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.794+0000 7f4a77fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a7809e190 0x7f4a7812fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35542/0 (socket says 172.21.15.102:35542) 2024-09-13T17:42:16.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.794+0000 7f4a777fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4a780689d0 0x7f4a7809d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37530/0 (socket says 172.21.15.102:37530) 2024-09-13T17:42:16.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.794+0000 7f4a76ffd640 1 -- 172.21.15.102:0/206950963 learned_addr learned my addr 172.21.15.102:0/206950963 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:42:16.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.795+0000 7f4a77fff640 1 -- 172.21.15.102:0/206950963 >> v2:172.21.15.102:3300/0 conn(0x7f4a780689d0 msgr2=0x7f4a7809d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:16.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.795+0000 7f4a77fff640 1 --2- 172.21.15.102:0/206950963 >> v2:172.21.15.102:3300/0 conn(0x7f4a780689d0 0x7f4a7809d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.795+0000 7f4a77fff640 1 -- 172.21.15.102:0/206950963 >> v1:172.21.15.102:6789/0 conn(0x7f4a7809dd70 legacy=0x7f4a7812f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:16.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.795+0000 7f4a77fff640 1 -- 172.21.15.102:0/206950963 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a780696e0 con 0x7f4a7809e190 2024-09-13T17:42:16.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.795+0000 7f4a767fc640 1 -- 172.21.15.102:0/206950963 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1042321810 0 0) 0x7f4a78096b10 con 0x7f4a7809dd70 2024-09-13T17:42:16.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.795+0000 7f4a777fe640 1 --2- 172.21.15.102:0/206950963 >> v2:172.21.15.102:3300/0 conn(0x7f4a780689d0 0x7f4a7809d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:16.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.795+0000 7f4a77fff640 1 --2- 172.21.15.102:0/206950963 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a7809e190 0x7f4a7812fd20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4a70009850 tx=0x7f4a700090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daddbab03aa3b983 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:16.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.795+0000 7f4a767fc640 1 -- 172.21.15.102:0/206950963 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7002ac80 con 0x7f4a7809e190 2024-09-13T17:42:16.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.795+0000 7f4a767fc640 1 -- 172.21.15.102:0/206950963 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a7002ade0 con 0x7f4a7809e190 2024-09-13T17:42:16.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.795+0000 7f4a767fc640 1 -- 172.21.15.102:0/206950963 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a70032670 con 0x7f4a7809e190 2024-09-13T17:42:16.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.796+0000 7f4a7e15d640 1 -- 172.21.15.102:0/206950963 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a7809e190 msgr2=0x7f4a7812fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:16.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.796+0000 7f4a7e15d640 1 --2- 172.21.15.102:0/206950963 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a7809e190 0x7f4a7812fd20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4a70009850 tx=0x7f4a700090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.796+0000 7f4a7e15d640 1 -- 172.21.15.102:0/206950963 shutdown_connections 2024-09-13T17:42:16.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.796+0000 7f4a7e15d640 1 --2- 172.21.15.102:0/206950963 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a7809e190 0x7f4a7812fd20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.796+0000 7f4a7e15d640 1 --2- 172.21.15.102:0/206950963 >> v2:172.21.15.102:3300/0 conn(0x7f4a780689d0 0x7f4a7809d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.796+0000 7f4a7e15d640 1 -- 172.21.15.102:0/206950963 >> 172.21.15.102:0/206950963 conn(0x7f4a7806f320 msgr2=0x7f4a780a2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:16.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.796+0000 7f4a7e15d640 1 -- 172.21.15.102:0/206950963 shutdown_connections 2024-09-13T17:42:16.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.796+0000 7f4a7e15d640 1 -- 172.21.15.102:0/206950963 wait complete. 2024-09-13T17:42:16.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.797+0000 7f4a7e15d640 1 Processor -- start 2024-09-13T17:42:16.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.797+0000 7f4a7e15d640 1 -- start start 2024-09-13T17:42:16.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.797+0000 7f4a7e15d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a780689d0 0x7f4a78122620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:16.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.797+0000 7f4a7e15d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4a7809dd70 0x7f4a7811f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:16.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.797+0000 7f4a7e15d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a7809e190 0x7f4a7811f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:16.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.797+0000 7f4a7e15d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7809ebc0 con 0x7f4a7809e190 2024-09-13T17:42:16.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.797+0000 7f4a7e15d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7809ca30 con 0x7f4a780689d0 2024-09-13T17:42:16.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.797+0000 7f4a7e15d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a78121b80 con 0x7f4a7809dd70 2024-09-13T17:42:16.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.797+0000 7f4a777fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a780689d0 0x7f4a78122620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:16.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.797+0000 7f4a76ffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4a7809dd70 0x7f4a7811f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:16.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.797+0000 7f4a77fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a7809e190 0x7f4a7811f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:16.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.797+0000 7f4a777fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a780689d0 0x7f4a78122620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35554/0 (socket says 172.21.15.102:35554) 2024-09-13T17:42:16.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.797+0000 7f4a777fe640 1 -- 172.21.15.102:0/688444783 learned_addr learned my addr 172.21.15.102:0/688444783 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:16.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.797+0000 7f4a76ffd640 1 --2- 172.21.15.102:0/688444783 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4a7809dd70 0x7f4a7811f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:46800/0 (socket says 172.21.15.102:46800) 2024-09-13T17:42:16.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.798+0000 7f4a77fff640 1 -- 172.21.15.102:0/688444783 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4a7809dd70 msgr2=0x7f4a7811f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:16.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.798+0000 7f4a77fff640 1 --2- 172.21.15.102:0/688444783 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4a7809dd70 0x7f4a7811f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.798+0000 7f4a77fff640 1 -- 172.21.15.102:0/688444783 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a780689d0 msgr2=0x7f4a78122620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:16.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.798+0000 7f4a77fff640 1 --2- 172.21.15.102:0/688444783 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a780689d0 0x7f4a78122620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.798+0000 7f4a77fff640 1 -- 172.21.15.102:0/688444783 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a60009a20 con 0x7f4a7809e190 2024-09-13T17:42:16.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.798+0000 7f4a777fe640 1 --2- 172.21.15.102:0/688444783 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a780689d0 0x7f4a78122620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:16.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.798+0000 7f4a77fff640 1 --2- 172.21.15.102:0/688444783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a7809e190 0x7f4a7811f5b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4a70031000 tx=0x7f4a70032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:16.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.798+0000 7f4a6f7fe640 1 -- 172.21.15.102:0/688444783 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a70032e20 con 0x7f4a7809e190 2024-09-13T17:42:16.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.798+0000 7f4a6f7fe640 1 -- 172.21.15.102:0/688444783 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a70031a60 con 0x7f4a7809e190 2024-09-13T17:42:16.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.798+0000 7f4a7e15d640 1 -- 172.21.15.102:0/688444783 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a70009390 con 0x7f4a7809e190 2024-09-13T17:42:16.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.798+0000 7f4a6f7fe640 1 -- 172.21.15.102:0/688444783 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a70031e00 con 0x7f4a7809e190 2024-09-13T17:42:16.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.798+0000 7f4a7e15d640 1 -- 172.21.15.102:0/688444783 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a7811ee20 con 0x7f4a7809e190 2024-09-13T17:42:16.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.799+0000 7f4a6f7fe640 1 -- 172.21.15.102:0/688444783 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f4a70039040 con 0x7f4a7809e190 2024-09-13T17:42:16.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.799+0000 7f4a6f7fe640 1 -- 172.21.15.102:0/688444783 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7f4a70061d50 con 0x7f4a7809e190 2024-09-13T17:42:16.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.799+0000 7f4a7e15d640 1 -- 172.21.15.102:0/688444783 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a78121da0 con 0x7f4a7809e190 2024-09-13T17:42:16.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.804+0000 7f4a6f7fe640 1 -- 172.21.15.102:0/688444783 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f4a7003b3f0 con 0x7f4a7809e190 2024-09-13T17:42:16.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.955+0000 7f4a7e15d640 1 -- 172.21.15.102:0/688444783 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4a78064000 con 0x7f4a7809e190 2024-09-13T17:42:16.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.956+0000 7f4a6f7fe640 1 -- 172.21.15.102:0/688444783 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4a70042070 con 0x7f4a7809e190 2024-09-13T17:42:16.958 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-09-13T17:42:16.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.957+0000 7f4a7e15d640 1 -- 172.21.15.102:0/688444783 >> v1:172.21.15.102:6816/34937 conn(0x7f4a58031160 legacy=0x7f4a58033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:16.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.957+0000 7f4a7e15d640 1 -- 172.21.15.102:0/688444783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a7809e190 msgr2=0x7f4a7811f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:16.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.957+0000 7f4a7e15d640 1 --2- 172.21.15.102:0/688444783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a7809e190 0x7f4a7811f5b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4a70031000 tx=0x7f4a70032000 comp rx=0 tx=0).stop 2024-09-13T17:42:16.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.957+0000 7f4a7e15d640 1 -- 172.21.15.102:0/688444783 shutdown_connections 2024-09-13T17:42:16.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.958+0000 7f4a7e15d640 1 --2- 172.21.15.102:0/688444783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a7809e190 0x7f4a7811f5b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.958+0000 7f4a7e15d640 1 --2- 172.21.15.102:0/688444783 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4a7809dd70 0x7f4a7811f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.958+0000 7f4a7e15d640 1 --2- 172.21.15.102:0/688444783 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4a780689d0 0x7f4a78122620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.958+0000 7f4a7e15d640 1 -- 172.21.15.102:0/688444783 >> 172.21.15.102:0/688444783 conn(0x7f4a7806f320 msgr2=0x7f4a780a18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:16.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.958+0000 7f4a7e15d640 1 -- 172.21.15.102:0/688444783 shutdown_connections 2024-09-13T17:42:16.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:16.958+0000 7f4a7e15d640 1 -- 172.21.15.102:0/688444783 wait complete. 2024-09-13T17:42:16.970 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:42:17.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.060+0000 7fa282d1d640 1 Processor -- start 2024-09-13T17:42:17.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.061+0000 7fa282d1d640 1 -- start start 2024-09-13T17:42:17.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.061+0000 7fa282d1d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa27c09add0 0x7fa27c09b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.061+0000 7fa282d1d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa27c12d2f0 0x7fa27c12d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.061+0000 7fa282d1d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa27c05e2f0 con 0x7fa27c12dc20 2024-09-13T17:42:17.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.061+0000 7fa282d1d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c06de30 con 0x7fa27c09add0 2024-09-13T17:42:17.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.061+0000 7fa282d1d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c06c710 con 0x7fa27c12d2f0 2024-09-13T17:42:17.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.061+0000 7fa27bfff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa27c12d2f0 0x7fa27c12d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.061+0000 7fa280ab9640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa27c09add0 0x7fa27c09b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.061+0000 7fa27bfff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa27c12d2f0 0x7fa27c12d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:46808/0 (socket says 172.21.15.102:46808) 2024-09-13T17:42:17.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.061+0000 7fa27bfff640 1 -- 172.21.15.102:0/3655661233 learned_addr learned my addr 172.21.15.102:0/3655661233 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:17.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.061+0000 7fa2812ba640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fa27c12dc20 0x7fa27c134630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:60938/0 (socket says 172.21.15.102:60938) 2024-09-13T17:42:17.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.061+0000 7fa27bfff640 1 -- 172.21.15.102:0/3655661233 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa27c09add0 msgr2=0x7fa27c09b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:17.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.061+0000 7fa27bfff640 1 --2- 172.21.15.102:0/3655661233 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa27c09add0 0x7fa27c09b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.061+0000 7fa27bfff640 1 -- 172.21.15.102:0/3655661233 >> v1:172.21.15.102:6789/0 conn(0x7fa27c12dc20 legacy=0x7fa27c134630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.061+0000 7fa27bfff640 1 -- 172.21.15.102:0/3655661233 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa27c06d4b0 con 0x7fa27c12d2f0 2024-09-13T17:42:17.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.061+0000 7fa27b7fe640 1 -- 172.21.15.102:0/3655661233 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1575973137 0 0) 0x7fa27c05e2f0 con 0x7fa27c12dc20 2024-09-13T17:42:17.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.062+0000 7fa27bfff640 1 --2- 172.21.15.102:0/3655661233 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa27c12d2f0 0x7fa27c12d6e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa26c009f80 tx=0x7fa26c009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=206c8c83eb488d11 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:17.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.062+0000 7fa27b7fe640 1 -- 172.21.15.102:0/3655661233 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c031400 con 0x7fa27c12d2f0 2024-09-13T17:42:17.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.062+0000 7fa27b7fe640 1 -- 172.21.15.102:0/3655661233 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26c02d050 con 0x7fa27c12d2f0 2024-09-13T17:42:17.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.062+0000 7fa27b7fe640 1 -- 172.21.15.102:0/3655661233 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c031790 con 0x7fa27c12d2f0 2024-09-13T17:42:17.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.063+0000 7fa282d1d640 1 -- 172.21.15.102:0/3655661233 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa27c12d2f0 msgr2=0x7fa27c12d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.063+0000 7fa282d1d640 1 --2- 172.21.15.102:0/3655661233 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa27c12d2f0 0x7fa27c12d6e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa26c009f80 tx=0x7fa26c009840 comp rx=0 tx=0).stop 2024-09-13T17:42:17.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.063+0000 7fa282d1d640 1 -- 172.21.15.102:0/3655661233 shutdown_connections 2024-09-13T17:42:17.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.063+0000 7fa282d1d640 1 --2- 172.21.15.102:0/3655661233 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa27c12d2f0 0x7fa27c12d6e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.063+0000 7fa282d1d640 1 --2- 172.21.15.102:0/3655661233 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa27c09add0 0x7fa27c09b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.063+0000 7fa282d1d640 1 -- 172.21.15.102:0/3655661233 >> 172.21.15.102:0/3655661233 conn(0x7fa27c09f9d0 msgr2=0x7fa27c09ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:17.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.063+0000 7fa282d1d640 1 -- 172.21.15.102:0/3655661233 shutdown_connections 2024-09-13T17:42:17.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.063+0000 7fa282d1d640 1 -- 172.21.15.102:0/3655661233 wait complete. 2024-09-13T17:42:17.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.064+0000 7fa282d1d640 1 Processor -- start 2024-09-13T17:42:17.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.064+0000 7fa282d1d640 1 -- start start 2024-09-13T17:42:17.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.064+0000 7fa282d1d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa27c09add0 0x7fa27c129e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.064+0000 7fa282d1d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa27c12d2f0 0x7fa27c12a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.064+0000 7fa282d1d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa27c12dc20 0x7fa27c12a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.064+0000 7fa282d1d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c09d720 con 0x7fa27c09add0 2024-09-13T17:42:17.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.064+0000 7fa282d1d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c088390 con 0x7fa27c12dc20 2024-09-13T17:42:17.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.064+0000 7fa282d1d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c095af0 con 0x7fa27c12d2f0 2024-09-13T17:42:17.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.064+0000 7fa280ab9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa27c09add0 0x7fa27c129e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.064+0000 7fa280ab9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa27c09add0 0x7fa27c129e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37544/0 (socket says 172.21.15.102:37544) 2024-09-13T17:42:17.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.064+0000 7fa280ab9640 1 -- 172.21.15.102:0/3237589691 learned_addr learned my addr 172.21.15.102:0/3237589691 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:17.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.064+0000 7fa27bfff640 1 --2- 172.21.15.102:0/3237589691 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa27c12d2f0 0x7fa27c12a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.064+0000 7fa2812ba640 1 --2- 172.21.15.102:0/3237589691 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa27c12dc20 0x7fa27c12a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.065+0000 7fa280ab9640 1 -- 172.21.15.102:0/3237589691 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa27c12d2f0 msgr2=0x7fa27c12a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.065+0000 7fa280ab9640 1 --2- 172.21.15.102:0/3237589691 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa27c12d2f0 0x7fa27c12a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.065+0000 7fa280ab9640 1 -- 172.21.15.102:0/3237589691 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa27c12dc20 msgr2=0x7fa27c12a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.065+0000 7fa280ab9640 1 --2- 172.21.15.102:0/3237589691 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa27c12dc20 0x7fa27c12a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.065+0000 7fa280ab9640 1 -- 172.21.15.102:0/3237589691 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa26c009ae0 con 0x7fa27c09add0 2024-09-13T17:42:17.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.065+0000 7fa2812ba640 1 --2- 172.21.15.102:0/3237589691 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa27c12dc20 0x7fa27c12a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:17.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.065+0000 7fa27bfff640 1 --2- 172.21.15.102:0/3237589691 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa27c12d2f0 0x7fa27c12a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:17.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.065+0000 7fa280ab9640 1 --2- 172.21.15.102:0/3237589691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa27c09add0 0x7fa27c129e10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa27400ea20 tx=0x7fa2740090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:17.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.065+0000 7fa278ff9640 1 -- 172.21.15.102:0/3237589691 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27400c9b0 con 0x7fa27c09add0 2024-09-13T17:42:17.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.065+0000 7fa278ff9640 1 -- 172.21.15.102:0/3237589691 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa27400cb10 con 0x7fa27c09add0 2024-09-13T17:42:17.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.065+0000 7fa282d1d640 1 -- 172.21.15.102:0/3237589691 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa27c088d00 con 0x7fa27c09add0 2024-09-13T17:42:17.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.065+0000 7fa278ff9640 1 -- 172.21.15.102:0/3237589691 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27400ce90 con 0x7fa27c09add0 2024-09-13T17:42:17.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.065+0000 7fa282d1d640 1 -- 172.21.15.102:0/3237589691 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa27c12c260 con 0x7fa27c09add0 2024-09-13T17:42:17.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.066+0000 7fa278ff9640 1 -- 172.21.15.102:0/3237589691 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fa274013680 con 0x7fa27c09add0 2024-09-13T17:42:17.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.066+0000 7fa282d1d640 1 -- 172.21.15.102:0/3237589691 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa27c064000 con 0x7fa27c09add0 2024-09-13T17:42:17.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.066+0000 7fa278ff9640 1 -- 172.21.15.102:0/3237589691 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7fa274042390 con 0x7fa27c09add0 2024-09-13T17:42:17.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.070+0000 7fa278ff9640 1 -- 172.21.15.102:0/3237589691 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fa274021020 con 0x7fa27c09add0 2024-09-13T17:42:17.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.254+0000 7fa282d1d640 1 -- 172.21.15.102:0/3237589691 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa27c12b9d0 con 0x7fa27c09add0 2024-09-13T17:42:17.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.254+0000 7fa278ff9640 1 -- 172.21.15.102:0/3237589691 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2266 (secure 0 0 0) 0x7fa274023ad0 con 0x7fa27c09add0 2024-09-13T17:42:17.256 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:42:17.256 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:42:06.611970+0000","modified":"2024-09-13T17:42:09.305093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4210":{"gid":4210,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6817/3443280887","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":3443280887}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-09-13T17:42:17.256 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 5 2024-09-13T17:42:17.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.256+0000 7fa282d1d640 1 -- 172.21.15.102:0/3237589691 >> v1:172.21.15.102:6816/34937 conn(0x7fa25c031190 legacy=0x7fa25c033640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.256+0000 7fa282d1d640 1 -- 172.21.15.102:0/3237589691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa27c09add0 msgr2=0x7fa27c129e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.256+0000 7fa282d1d640 1 --2- 172.21.15.102:0/3237589691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa27c09add0 0x7fa27c129e10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa27400ea20 tx=0x7fa2740090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.256+0000 7fa282d1d640 1 -- 172.21.15.102:0/3237589691 shutdown_connections 2024-09-13T17:42:17.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.256+0000 7fa282d1d640 1 --2- 172.21.15.102:0/3237589691 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa27c12dc20 0x7fa27c12a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.256+0000 7fa282d1d640 1 --2- 172.21.15.102:0/3237589691 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa27c12d2f0 0x7fa27c12a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.256+0000 7fa282d1d640 1 --2- 172.21.15.102:0/3237589691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa27c09add0 0x7fa27c129e10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.256+0000 7fa282d1d640 1 -- 172.21.15.102:0/3237589691 >> 172.21.15.102:0/3237589691 conn(0x7fa27c09f9d0 msgr2=0x7fa27c098a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:17.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.256+0000 7fa282d1d640 1 -- 172.21.15.102:0/3237589691 shutdown_connections 2024-09-13T17:42:17.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.256+0000 7fa282d1d640 1 -- 172.21.15.102:0/3237589691 wait complete. 2024-09-13T17:42:17.269 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:17.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.362+0000 7f5ffd00f640 1 Processor -- start 2024-09-13T17:42:17.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.362+0000 7f5ffd00f640 1 -- start start 2024-09-13T17:42:17.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.362+0000 7f5ffd00f640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5ff806b140 0x7f5ff80998c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.362+0000 7f5ffd00f640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5ff809ac20 0x7f5ff8099e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.362+0000 7f5ffd00f640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ff809a340 0x7f5ff812d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.362+0000 7f5ffd00f640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff806a680 con 0x7f5ff809ac20 2024-09-13T17:42:17.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.362+0000 7f5ffd00f640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff806ee90 con 0x7f5ff809a340 2024-09-13T17:42:17.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.363+0000 7f5ffd00f640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff806ac90 con 0x7f5ff806b140 2024-09-13T17:42:17.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.363+0000 7f5ff759e640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ff809a340 0x7f5ff812d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.363+0000 7f5ff6d9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5ff806b140 0x7f5ff80998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.363+0000 7f5fee59c640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5ff809ac20 0x7f5ff8099e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.363+0000 7f5ff759e640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ff809a340 0x7f5ff812d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35582/0 (socket says 172.21.15.102:35582) 2024-09-13T17:42:17.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.363+0000 7f5ff759e640 1 -- 172.21.15.102:0/3856999369 learned_addr learned my addr 172.21.15.102:0/3856999369 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:17.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.363+0000 7f5ff6d9d640 1 --2- 172.21.15.102:0/3856999369 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5ff806b140 0x7f5ff80998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:46826/0 (socket says 172.21.15.102:46826) 2024-09-13T17:42:17.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.363+0000 7f5ff759e640 1 -- 172.21.15.102:0/3856999369 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5ff806b140 msgr2=0x7f5ff80998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:17.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.363+0000 7f5ff759e640 1 --2- 172.21.15.102:0/3856999369 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5ff806b140 0x7f5ff80998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.363+0000 7f5ff759e640 1 -- 172.21.15.102:0/3856999369 >> v2:172.21.15.102:3300/0 conn(0x7f5ff809ac20 msgr2=0x7f5ff8099e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:17.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.363+0000 7f5ff759e640 1 --2- 172.21.15.102:0/3856999369 >> v2:172.21.15.102:3300/0 conn(0x7f5ff809ac20 0x7f5ff8099e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.363+0000 7f5ff759e640 1 -- 172.21.15.102:0/3856999369 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff8099510 con 0x7f5ff809a340 2024-09-13T17:42:17.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.363+0000 7f5fee59c640 1 --2- 172.21.15.102:0/3856999369 >> v2:172.21.15.102:3300/0 conn(0x7f5ff809ac20 0x7f5ff8099e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:17.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.364+0000 7f5ff759e640 1 --2- 172.21.15.102:0/3856999369 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ff809a340 0x7f5ff812d280 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009f50 tx=0x7f5ff0009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=906221f673c15466 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:17.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.364+0000 7f5ff659c640 1 -- 172.21.15.102:0/3856999369 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0031400 con 0x7f5ff809a340 2024-09-13T17:42:17.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.364+0000 7f5ff659c640 1 -- 172.21.15.102:0/3856999369 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff002d050 con 0x7f5ff809a340 2024-09-13T17:42:17.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.364+0000 7f5ff659c640 1 -- 172.21.15.102:0/3856999369 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff00317d0 con 0x7f5ff809a340 2024-09-13T17:42:17.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.364+0000 7f5ffd00f640 1 -- 172.21.15.102:0/3856999369 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ff809a340 msgr2=0x7f5ff812d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.364+0000 7f5ffd00f640 1 --2- 172.21.15.102:0/3856999369 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ff809a340 0x7f5ff812d280 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009f50 tx=0x7f5ff0009840 comp rx=0 tx=0).stop 2024-09-13T17:42:17.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.364+0000 7f5ffd00f640 1 -- 172.21.15.102:0/3856999369 shutdown_connections 2024-09-13T17:42:17.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.364+0000 7f5ffd00f640 1 --2- 172.21.15.102:0/3856999369 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ff809a340 0x7f5ff812d280 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.364+0000 7f5ffd00f640 1 --2- 172.21.15.102:0/3856999369 >> v2:172.21.15.102:3300/0 conn(0x7f5ff809ac20 0x7f5ff8099e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.364+0000 7f5ffd00f640 1 --2- 172.21.15.102:0/3856999369 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5ff806b140 0x7f5ff80998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.364+0000 7f5ffd00f640 1 -- 172.21.15.102:0/3856999369 >> 172.21.15.102:0/3856999369 conn(0x7f5ff809e820 msgr2=0x7f5ff809ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:17.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.364+0000 7f5ffd00f640 1 -- 172.21.15.102:0/3856999369 shutdown_connections 2024-09-13T17:42:17.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.364+0000 7f5ffd00f640 1 -- 172.21.15.102:0/3856999369 wait complete. 2024-09-13T17:42:17.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.365+0000 7f5ffd00f640 1 Processor -- start 2024-09-13T17:42:17.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.365+0000 7f5ffd00f640 1 -- start start 2024-09-13T17:42:17.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.366+0000 7f5ffd00f640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ff806b140 0x7f5ff80970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.366+0000 7f5ffd00f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ff809a340 0x7f5ff8097600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.366+0000 7f5ffd00f640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5ff809ac20 0x7f5ff8097b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.366+0000 7f5ffd00f640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff809dae0 con 0x7f5ff809a340 2024-09-13T17:42:17.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.366+0000 7f5ffd00f640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff8069dd0 con 0x7f5ff806b140 2024-09-13T17:42:17.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.366+0000 7f5ffd00f640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff8120b80 con 0x7f5ff809ac20 2024-09-13T17:42:17.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.366+0000 7f5ff6d9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ff806b140 0x7f5ff80970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.366+0000 7f5ff759e640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5ff809ac20 0x7f5ff8097b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.366+0000 7f5fee59c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ff809a340 0x7f5ff8097600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.366+0000 7f5ff6d9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ff806b140 0x7f5ff80970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35598/0 (socket says 172.21.15.102:35598) 2024-09-13T17:42:17.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.366+0000 7f5ff6d9d640 1 -- 172.21.15.102:0/3850951897 learned_addr learned my addr 172.21.15.102:0/3850951897 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:17.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.366+0000 7f5ff759e640 1 -- 172.21.15.102:0/3850951897 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ff806b140 msgr2=0x7f5ff80970c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.366+0000 7f5ff759e640 1 --2- 172.21.15.102:0/3850951897 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ff806b140 0x7f5ff80970c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.366+0000 7f5ff759e640 1 -- 172.21.15.102:0/3850951897 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ff809a340 msgr2=0x7f5ff8097600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.366+0000 7f5ff759e640 1 --2- 172.21.15.102:0/3850951897 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ff809a340 0x7f5ff8097600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.366+0000 7f5ff759e640 1 -- 172.21.15.102:0/3850951897 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe0009390 con 0x7f5ff809ac20 2024-09-13T17:42:17.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.366+0000 7f5fee59c640 1 --2- 172.21.15.102:0/3850951897 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ff809a340 0x7f5ff8097600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:17.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.366+0000 7f5ff6d9d640 1 --2- 172.21.15.102:0/3850951897 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ff806b140 0x7f5ff80970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:17.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.366+0000 7f5ff759e640 1 --2- 172.21.15.102:0/3850951897 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5ff809ac20 0x7f5ff8097b40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0030000 tx=0x7f5ff0036000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:17.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.366+0000 7f5fef7fe640 1 -- 172.21.15.102:0/3850951897 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0007170 con 0x7f5ff809ac20 2024-09-13T17:42:17.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.366+0000 7f5fef7fe640 1 -- 172.21.15.102:0/3850951897 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff0008e50 con 0x7f5ff809ac20 2024-09-13T17:42:17.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.366+0000 7f5ffd00f640 1 -- 172.21.15.102:0/3850951897 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff0009ae0 con 0x7f5ff809ac20 2024-09-13T17:42:17.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.366+0000 7f5fef7fe640 1 -- 172.21.15.102:0/3850951897 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff003b660 con 0x7f5ff809ac20 2024-09-13T17:42:17.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.367+0000 7f5ffd00f640 1 -- 172.21.15.102:0/3850951897 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff8098580 con 0x7f5ff809ac20 2024-09-13T17:42:17.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.367+0000 7f5fef7fe640 1 -- 172.21.15.102:0/3850951897 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f5ff0038050 con 0x7f5ff809ac20 2024-09-13T17:42:17.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.367+0000 7f5fef7fe640 1 -- 172.21.15.102:0/3850951897 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7f5ff0062c30 con 0x7f5ff809ac20 2024-09-13T17:42:17.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.368+0000 7f5ffd00f640 1 -- 172.21.15.102:0/3850951897 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff8120da0 con 0x7f5ff809ac20 2024-09-13T17:42:17.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.372+0000 7f5fef7fe640 1 -- 172.21.15.102:0/3850951897 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f5ff8120da0 con 0x7f5ff809ac20 2024-09-13T17:42:17.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.526+0000 7f5ffd00f640 1 -- 172.21.15.102:0/3850951897 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ff8120da0 con 0x7f5ff809ac20 2024-09-13T17:42:17.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.527+0000 7f5fef7fe640 1 -- 172.21.15.102:0/3850951897 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+11375 (secure 0 0 0) 0x7f5ff8120da0 con 0x7f5ff809ac20 2024-09-13T17:42:17.529 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:42:17.529 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":20,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:42:07.375548+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:04.672265+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:05.583134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:17.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.529+0000 7f5ffd00f640 1 -- 172.21.15.102:0/3850951897 >> v1:172.21.15.102:6816/34937 conn(0x7f5fd8031160 legacy=0x7f5fd8033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.529+0000 7f5ffd00f640 1 -- 172.21.15.102:0/3850951897 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5ff809ac20 msgr2=0x7f5ff8097b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.529+0000 7f5ffd00f640 1 --2- 172.21.15.102:0/3850951897 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5ff809ac20 0x7f5ff8097b40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0030000 tx=0x7f5ff0036000 comp rx=0 tx=0).stop 2024-09-13T17:42:17.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.529+0000 7f5ffd00f640 1 -- 172.21.15.102:0/3850951897 shutdown_connections 2024-09-13T17:42:17.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.529+0000 7f5ffd00f640 1 --2- 172.21.15.102:0/3850951897 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5ff809ac20 0x7f5ff8097b40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.529+0000 7f5ffd00f640 1 --2- 172.21.15.102:0/3850951897 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ff809a340 0x7f5ff8097600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.529+0000 7f5ffd00f640 1 --2- 172.21.15.102:0/3850951897 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ff806b140 0x7f5ff80970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.529+0000 7f5ffd00f640 1 -- 172.21.15.102:0/3850951897 >> 172.21.15.102:0/3850951897 conn(0x7f5ff809e820 msgr2=0x7f5ff809c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:17.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.529+0000 7f5ffd00f640 1 -- 172.21.15.102:0/3850951897 shutdown_connections 2024-09-13T17:42:17.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.529+0000 7f5ffd00f640 1 -- 172.21.15.102:0/3850951897 wait complete. 2024-09-13T17:42:17.542 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:54.442612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T17:41:59.076782+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 16, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:42:04.672265+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:42:05.583134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:42:17.542 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:42:17.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.636+0000 7f82dfcc9640 1 Processor -- start 2024-09-13T17:42:17.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.636+0000 7f82dfcc9640 1 -- start start 2024-09-13T17:42:17.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.636+0000 7f82dfcc9640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f82d809c510 0x7f82d809a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.637+0000 7f82dfcc9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f82d809b220 0x7f82d81377f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.637+0000 7f82dfcc9640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82d806f740 con 0x7f82d809ae50 2024-09-13T17:42:17.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.637+0000 7f82dfcc9640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d8095f90 con 0x7f82d809c510 2024-09-13T17:42:17.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.637+0000 7f82dfcc9640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f82d809c8e0 con 0x7f82d809b220 2024-09-13T17:42:17.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.637+0000 7f82dda65640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f82d809c510 0x7f82d809a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.637+0000 7f82dd264640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f82d809ae50 0x7f82d81340c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:60942/0 (socket says 172.21.15.102:60942) 2024-09-13T17:42:17.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.637+0000 7f82dd264640 1 -- 172.21.15.102:0/2437924338 learned_addr learned my addr 172.21.15.102:0/2437924338 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:42:17.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.637+0000 7f82de266640 1 --2- 172.21.15.102:0/2437924338 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f82d809b220 0x7f82d81377f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.637+0000 7f82dca63640 1 -- 172.21.15.102:0/2437924338 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3222799716 0 0) 0x7f82d806f740 con 0x7f82d809ae50 2024-09-13T17:42:17.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.637+0000 7f82de266640 1 -- 172.21.15.102:0/2437924338 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f82d809c510 msgr2=0x7f82d809a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:17.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.637+0000 7f82de266640 1 --2- 172.21.15.102:0/2437924338 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f82d809c510 0x7f82d809a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.637+0000 7f82de266640 1 -- 172.21.15.102:0/2437924338 >> v1:172.21.15.102:6789/0 conn(0x7f82d809ae50 legacy=0x7f82d81340c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.637+0000 7f82de266640 1 -- 172.21.15.102:0/2437924338 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d8096b00 con 0x7f82d809b220 2024-09-13T17:42:17.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.637+0000 7f82dda65640 1 --2- 172.21.15.102:0/2437924338 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f82d809c510 0x7f82d809a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:17.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.638+0000 7f82de266640 1 --2- 172.21.15.102:0/2437924338 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f82d809b220 0x7f82d81377f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f82d0009e90 tx=0x7f82d0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cbb6cc4d6d6f121 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:17.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.638+0000 7f82dca63640 1 -- 172.21.15.102:0/2437924338 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d0007550 con 0x7f82d809b220 2024-09-13T17:42:17.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.638+0000 7f82dca63640 1 -- 172.21.15.102:0/2437924338 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d002d050 con 0x7f82d809b220 2024-09-13T17:42:17.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.638+0000 7f82dca63640 1 -- 172.21.15.102:0/2437924338 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d00078e0 con 0x7f82d809b220 2024-09-13T17:42:17.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.638+0000 7f82dfcc9640 1 -- 172.21.15.102:0/2437924338 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f82d809b220 msgr2=0x7f82d81377f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.638+0000 7f82dfcc9640 1 --2- 172.21.15.102:0/2437924338 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f82d809b220 0x7f82d81377f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f82d0009e90 tx=0x7f82d0009780 comp rx=0 tx=0).stop 2024-09-13T17:42:17.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.638+0000 7f82dfcc9640 1 -- 172.21.15.102:0/2437924338 shutdown_connections 2024-09-13T17:42:17.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.638+0000 7f82dfcc9640 1 --2- 172.21.15.102:0/2437924338 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f82d809b220 0x7f82d81377f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.638+0000 7f82dfcc9640 1 --2- 172.21.15.102:0/2437924338 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f82d809c510 0x7f82d809a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.638+0000 7f82dfcc9640 1 -- 172.21.15.102:0/2437924338 >> 172.21.15.102:0/2437924338 conn(0x7f82d809f960 msgr2=0x7f82d809ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:17.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.638+0000 7f82dfcc9640 1 -- 172.21.15.102:0/2437924338 shutdown_connections 2024-09-13T17:42:17.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.639+0000 7f82dfcc9640 1 -- 172.21.15.102:0/2437924338 wait complete. 2024-09-13T17:42:17.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.639+0000 7f82dfcc9640 1 Processor -- start 2024-09-13T17:42:17.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.639+0000 7f82dfcc9640 1 -- start start 2024-09-13T17:42:17.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.640+0000 7f82dfcc9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82d809ae50 0x7f82d8094020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.640+0000 7f82dfcc9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f82d809b220 0x7f82d80900e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.640+0000 7f82dfcc9640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f82d809c510 0x7f82d8090620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.640+0000 7f82dfcc9640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d8092150 con 0x7f82d809ae50 2024-09-13T17:42:17.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.640+0000 7f82dfcc9640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d8095340 con 0x7f82d809c510 2024-09-13T17:42:17.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.640+0000 7f82dfcc9640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f82d8091550 con 0x7f82d809b220 2024-09-13T17:42:17.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.640+0000 7f82dda65640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82d809ae50 0x7f82d8094020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.640+0000 7f82de266640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f82d809c510 0x7f82d8090620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.640+0000 7f82dd264640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f82d809b220 0x7f82d80900e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.640+0000 7f82dda65640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82d809ae50 0x7f82d8094020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37564/0 (socket says 172.21.15.102:37564) 2024-09-13T17:42:17.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.640+0000 7f82dda65640 1 -- 172.21.15.102:0/1363194967 learned_addr learned my addr 172.21.15.102:0/1363194967 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:17.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.640+0000 7f82dda65640 1 -- 172.21.15.102:0/1363194967 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f82d809b220 msgr2=0x7f82d80900e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.640+0000 7f82dda65640 1 --2- 172.21.15.102:0/1363194967 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f82d809b220 0x7f82d80900e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.640+0000 7f82dda65640 1 -- 172.21.15.102:0/1363194967 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f82d809c510 msgr2=0x7f82d8090620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.640+0000 7f82dda65640 1 --2- 172.21.15.102:0/1363194967 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f82d809c510 0x7f82d8090620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.640+0000 7f82dda65640 1 -- 172.21.15.102:0/1363194967 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d4009390 con 0x7f82d809ae50 2024-09-13T17:42:17.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.640+0000 7f82de266640 1 --2- 172.21.15.102:0/1363194967 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f82d809c510 0x7f82d8090620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:17.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.640+0000 7f82dda65640 1 --2- 172.21.15.102:0/1363194967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82d809ae50 0x7f82d8094020 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f82d400ecb0 tx=0x7f82d40090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:17.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.640+0000 7f82cdffb640 1 -- 172.21.15.102:0/1363194967 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d400cbc0 con 0x7f82d809ae50 2024-09-13T17:42:17.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.640+0000 7f82cdffb640 1 -- 172.21.15.102:0/1363194967 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d400cd20 con 0x7f82d809ae50 2024-09-13T17:42:17.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.640+0000 7f82dfcc9640 1 -- 172.21.15.102:0/1363194967 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82d0009a20 con 0x7f82d809ae50 2024-09-13T17:42:17.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.640+0000 7f82cdffb640 1 -- 172.21.15.102:0/1363194967 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d4013680 con 0x7f82d809ae50 2024-09-13T17:42:17.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.641+0000 7f82dfcc9640 1 -- 172.21.15.102:0/1363194967 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d8093840 con 0x7f82d809ae50 2024-09-13T17:42:17.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.642+0000 7f82cdffb640 1 -- 172.21.15.102:0/1363194967 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f82d40139f0 con 0x7f82d809ae50 2024-09-13T17:42:17.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.642+0000 7f82cdffb640 1 -- 172.21.15.102:0/1363194967 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7f82d4042440 con 0x7f82d809ae50 2024-09-13T17:42:17.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.642+0000 7f82dfcc9640 1 -- 172.21.15.102:0/1363194967 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82d8064000 con 0x7f82d809ae50 2024-09-13T17:42:17.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.646+0000 7f82cdffb640 1 -- 172.21.15.102:0/1363194967 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f82d4021070 con 0x7f82d809ae50 2024-09-13T17:42:17.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.798+0000 7f82dfcc9640 1 -- 172.21.15.102:0/1363194967 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f82d8098e80 con 0x7f82d809ae50 2024-09-13T17:42:17.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.798+0000 7f82cdffb640 1 -- 172.21.15.102:0/1363194967 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f82d4013ca0 con 0x7f82d809ae50 2024-09-13T17:42:17.800 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-09-13T17:42:17.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.800+0000 7f82dfcc9640 1 -- 172.21.15.102:0/1363194967 >> v1:172.21.15.102:6816/34937 conn(0x7f82bc031190 legacy=0x7f82bc033640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.800+0000 7f82dfcc9640 1 -- 172.21.15.102:0/1363194967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82d809ae50 msgr2=0x7f82d8094020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.800+0000 7f82dfcc9640 1 --2- 172.21.15.102:0/1363194967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82d809ae50 0x7f82d8094020 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f82d400ecb0 tx=0x7f82d40090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.800+0000 7f82dfcc9640 1 -- 172.21.15.102:0/1363194967 shutdown_connections 2024-09-13T17:42:17.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.800+0000 7f82dfcc9640 1 --2- 172.21.15.102:0/1363194967 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f82d809c510 0x7f82d8090620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.800+0000 7f82dfcc9640 1 --2- 172.21.15.102:0/1363194967 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f82d809b220 0x7f82d80900e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.800+0000 7f82dfcc9640 1 --2- 172.21.15.102:0/1363194967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82d809ae50 0x7f82d8094020 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.800+0000 7f82dfcc9640 1 -- 172.21.15.102:0/1363194967 >> 172.21.15.102:0/1363194967 conn(0x7f82d809f960 msgr2=0x7f82d806e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:17.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.800+0000 7f82dfcc9640 1 -- 172.21.15.102:0/1363194967 shutdown_connections 2024-09-13T17:42:17.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.800+0000 7f82dfcc9640 1 -- 172.21.15.102:0/1363194967 wait complete. 2024-09-13T17:42:17.813 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-13T17:42:17.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.907+0000 7fb3f7e34640 1 Processor -- start 2024-09-13T17:42:17.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.907+0000 7fb3f7e34640 1 -- start start 2024-09-13T17:42:17.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.908+0000 7fb3f7e34640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb3f009df40 0x7fb3f0134960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.908+0000 7fb3f7e34640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb3f0134ea0 0x7fb3f0137280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.908+0000 7fb3f7e34640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3f0096b10 con 0x7fb3f00689d0 2024-09-13T17:42:17.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.908+0000 7fb3f7e34640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3f0068da0 con 0x7fb3f009df40 2024-09-13T17:42:17.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.908+0000 7fb3f7e34640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f0096db0 con 0x7fb3f0134ea0 2024-09-13T17:42:17.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.908+0000 7fb3f53cf640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb3f009df40 0x7fb3f0134960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.908+0000 7fb3f63d1640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb3f0134ea0 0x7fb3f0137280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.908+0000 7fb3f5bd0640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fb3f00689d0 0x7fb3f009d830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:60944/0 (socket says 172.21.15.102:60944) 2024-09-13T17:42:17.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.908+0000 7fb3f63d1640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb3f0134ea0 0x7fb3f0137280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35642/0 (socket says 172.21.15.102:35642) 2024-09-13T17:42:17.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.908+0000 7fb3f5bd0640 1 -- 172.21.15.102:0/112745289 learned_addr learned my addr 172.21.15.102:0/112745289 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:42:17.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.908+0000 7fb3f53cf640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb3f009df40 0x7fb3f0134960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37570/0 (socket says 172.21.15.102:37570) 2024-09-13T17:42:17.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.908+0000 7fb3f53cf640 1 -- 172.21.15.102:0/112745289 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb3f0134ea0 msgr2=0x7fb3f0137280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:17.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.908+0000 7fb3f53cf640 1 --2- 172.21.15.102:0/112745289 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb3f0134ea0 0x7fb3f0137280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.908+0000 7fb3f53cf640 1 -- 172.21.15.102:0/112745289 >> v1:172.21.15.102:6789/0 conn(0x7fb3f00689d0 legacy=0x7fb3f009d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.908+0000 7fb3f53cf640 1 -- 172.21.15.102:0/112745289 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f00696e0 con 0x7fb3f009df40 2024-09-13T17:42:17.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.908+0000 7fb3f4bce640 1 -- 172.21.15.102:0/112745289 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2515099208 0 0) 0x7fb3f0096b10 con 0x7fb3f00689d0 2024-09-13T17:42:17.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.908+0000 7fb3f63d1640 1 --2- 172.21.15.102:0/112745289 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb3f0134ea0 0x7fb3f0137280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:17.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.909+0000 7fb3f53cf640 1 --2- 172.21.15.102:0/112745289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3f009df40 0x7fb3f0134960 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0009850 tx=0x7fb3e00090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d8701905ed52e46 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:17.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.909+0000 7fb3f4bce640 1 -- 172.21.15.102:0/112745289 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e002ac60 con 0x7fb3f009df40 2024-09-13T17:42:17.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.909+0000 7fb3f4bce640 1 -- 172.21.15.102:0/112745289 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e002adc0 con 0x7fb3f009df40 2024-09-13T17:42:17.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.909+0000 7fb3f4bce640 1 -- 172.21.15.102:0/112745289 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0032670 con 0x7fb3f009df40 2024-09-13T17:42:17.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.910+0000 7fb3f7e34640 1 -- 172.21.15.102:0/112745289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3f009df40 msgr2=0x7fb3f0134960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.910+0000 7fb3f7e34640 1 --2- 172.21.15.102:0/112745289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3f009df40 0x7fb3f0134960 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0009850 tx=0x7fb3e00090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.910+0000 7fb3f7e34640 1 -- 172.21.15.102:0/112745289 shutdown_connections 2024-09-13T17:42:17.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.910+0000 7fb3f7e34640 1 --2- 172.21.15.102:0/112745289 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb3f0134ea0 0x7fb3f0137280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.910+0000 7fb3f7e34640 1 --2- 172.21.15.102:0/112745289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3f009df40 0x7fb3f0134960 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.910+0000 7fb3f7e34640 1 -- 172.21.15.102:0/112745289 >> 172.21.15.102:0/112745289 conn(0x7fb3f006f320 msgr2=0x7fb3f00a2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:17.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.910+0000 7fb3f7e34640 1 -- 172.21.15.102:0/112745289 shutdown_connections 2024-09-13T17:42:17.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.910+0000 7fb3f7e34640 1 -- 172.21.15.102:0/112745289 wait complete. 2024-09-13T17:42:17.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.911+0000 7fb3f7e34640 1 Processor -- start 2024-09-13T17:42:17.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.911+0000 7fb3f7e34640 1 -- start start 2024-09-13T17:42:17.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.911+0000 7fb3f7e34640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3f00689d0 0x7fb3f011e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.911+0000 7fb3f7e34640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb3f009df40 0x7fb3f011a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.911+0000 7fb3f7e34640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb3f0134ea0 0x7fb3f011a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.911+0000 7fb3f7e34640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f011c090 con 0x7fb3f00689d0 2024-09-13T17:42:17.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.911+0000 7fb3f7e34640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f011d470 con 0x7fb3f009df40 2024-09-13T17:42:17.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.911+0000 7fb3f7e34640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f0130b40 con 0x7fb3f0134ea0 2024-09-13T17:42:17.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.911+0000 7fb3f5bd0640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3f00689d0 0x7fb3f011e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.911+0000 7fb3f53cf640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb3f009df40 0x7fb3f011a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.911+0000 7fb3f5bd0640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3f00689d0 0x7fb3f011e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37580/0 (socket says 172.21.15.102:37580) 2024-09-13T17:42:17.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.911+0000 7fb3f53cf640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb3f009df40 0x7fb3f011a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35658/0 (socket says 172.21.15.102:35658) 2024-09-13T17:42:17.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.911+0000 7fb3f5bd0640 1 -- 172.21.15.102:0/421396644 learned_addr learned my addr 172.21.15.102:0/421396644 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:17.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.911+0000 7fb3f63d1640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb3f0134ea0 0x7fb3f011a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.912+0000 7fb3f5bd0640 1 -- 172.21.15.102:0/421396644 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb3f0134ea0 msgr2=0x7fb3f011a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.912+0000 7fb3f5bd0640 1 --2- 172.21.15.102:0/421396644 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb3f0134ea0 0x7fb3f011a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.912+0000 7fb3f5bd0640 1 -- 172.21.15.102:0/421396644 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb3f009df40 msgr2=0x7fb3f011a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.912+0000 7fb3f5bd0640 1 --2- 172.21.15.102:0/421396644 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb3f009df40 0x7fb3f011a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.912+0000 7fb3f5bd0640 1 -- 172.21.15.102:0/421396644 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3ec009390 con 0x7fb3f00689d0 2024-09-13T17:42:17.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.912+0000 7fb3f5bd0640 1 --2- 172.21.15.102:0/421396644 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3f00689d0 0x7fb3f011e2b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb3e800ccc0 tx=0x7fb3e80019e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:17.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.912+0000 7fb3e5ffb640 1 -- 172.21.15.102:0/421396644 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e800ce80 con 0x7fb3f00689d0 2024-09-13T17:42:17.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.912+0000 7fb3e5ffb640 1 -- 172.21.15.102:0/421396644 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e800a850 con 0x7fb3f00689d0 2024-09-13T17:42:17.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.912+0000 7fb3e5ffb640 1 -- 172.21.15.102:0/421396644 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e800ac30 con 0x7fb3f00689d0 2024-09-13T17:42:17.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.912+0000 7fb3f7e34640 1 -- 172.21.15.102:0/421396644 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e0009390 con 0x7fb3f00689d0 2024-09-13T17:42:17.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.912+0000 7fb3f7e34640 1 -- 172.21.15.102:0/421396644 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f011d810 con 0x7fb3f00689d0 2024-09-13T17:42:17.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.913+0000 7fb3e5ffb640 1 -- 172.21.15.102:0/421396644 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fb3e8007300 con 0x7fb3f00689d0 2024-09-13T17:42:17.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.913+0000 7fb3e5ffb640 1 -- 172.21.15.102:0/421396644 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7fb3e80420f0 con 0x7fb3f00689d0 2024-09-13T17:42:17.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.913+0000 7fb3f7e34640 1 -- 172.21.15.102:0/421396644 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f0064000 con 0x7fb3f00689d0 2024-09-13T17:42:17.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:17.918+0000 7fb3e5ffb640 1 -- 172.21.15.102:0/421396644 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fb3e80075b0 con 0x7fb3f00689d0 2024-09-13T17:42:18.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.068+0000 7fb3f7e34640 1 -- 172.21.15.102:0/421396644 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb3f00a39c0 con 0x7fb3f00689d0 2024-09-13T17:42:18.071 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.069+0000 7fb3e5ffb640 1 -- 172.21.15.102:0/421396644 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb3e8010770 con 0x7fb3f00689d0 2024-09-13T17:42:18.071 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-09-13T17:42:18.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.071+0000 7fb3f7e34640 1 -- 172.21.15.102:0/421396644 >> v1:172.21.15.102:6816/34937 conn(0x7fb3d4031160 legacy=0x7fb3d4033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:18.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.071+0000 7fb3f7e34640 1 -- 172.21.15.102:0/421396644 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3f00689d0 msgr2=0x7fb3f011e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:18.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.071+0000 7fb3f7e34640 1 --2- 172.21.15.102:0/421396644 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3f00689d0 0x7fb3f011e2b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb3e800ccc0 tx=0x7fb3e80019e0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.071+0000 7fb3f7e34640 1 -- 172.21.15.102:0/421396644 shutdown_connections 2024-09-13T17:42:18.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.071+0000 7fb3f7e34640 1 --2- 172.21.15.102:0/421396644 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb3f0134ea0 0x7fb3f011a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.071+0000 7fb3f7e34640 1 --2- 172.21.15.102:0/421396644 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb3f009df40 0x7fb3f011a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.071+0000 7fb3f7e34640 1 --2- 172.21.15.102:0/421396644 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3f00689d0 0x7fb3f011e2b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.071+0000 7fb3f7e34640 1 -- 172.21.15.102:0/421396644 >> 172.21.15.102:0/421396644 conn(0x7fb3f006f320 msgr2=0x7fb3f00a18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:18.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.071+0000 7fb3f7e34640 1 -- 172.21.15.102:0/421396644 shutdown_connections 2024-09-13T17:42:18.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.071+0000 7fb3f7e34640 1 -- 172.21.15.102:0/421396644 wait complete. 2024-09-13T17:42:18.083 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:42:18.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.177+0000 7f5aa7ed4640 1 Processor -- start 2024-09-13T17:42:18.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.177+0000 7f5aa7ed4640 1 -- start start 2024-09-13T17:42:18.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.178+0000 7f5aa7ed4640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5aa009a620 0x7f5aa0096c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:18.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.178+0000 7f5aa7ed4640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5aa0097170 0x7f5aa0097540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:18.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.178+0000 7f5aa7ed4640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5aa012d180 0x7f5aa012d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:18.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.178+0000 7f5aa7ed4640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aa009a9f0 con 0x7f5aa0097170 2024-09-13T17:42:18.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.178+0000 7f5aa7ed4640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa00982a0 con 0x7f5aa009a620 2024-09-13T17:42:18.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.178+0000 7f5aa7ed4640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa0098040 con 0x7f5aa012d180 2024-09-13T17:42:18.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.178+0000 7f5aa546f640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5aa0097170 0x7f5aa0097540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:18.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.178+0000 7f5aa5c70640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5aa009a620 0x7f5aa0096c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:18.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.178+0000 7f5aa6471640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5aa012d180 0x7f5aa012d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:18.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.178+0000 7f5aa546f640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5aa0097170 0x7f5aa0097540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37596/0 (socket says 172.21.15.102:37596) 2024-09-13T17:42:18.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.178+0000 7f5aa546f640 1 -- 172.21.15.102:0/1649015188 learned_addr learned my addr 172.21.15.102:0/1649015188 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:18.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.178+0000 7f5aa5c70640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5aa009a620 0x7f5aa0096c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35666/0 (socket says 172.21.15.102:35666) 2024-09-13T17:42:18.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.178+0000 7f5aa546f640 1 -- 172.21.15.102:0/1649015188 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5aa012d180 msgr2=0x7f5aa012d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:18.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.178+0000 7f5aa546f640 1 --2- 172.21.15.102:0/1649015188 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5aa012d180 0x7f5aa012d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.178+0000 7f5aa546f640 1 -- 172.21.15.102:0/1649015188 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5aa009a620 msgr2=0x7f5aa0096c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:18.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.178+0000 7f5aa546f640 1 --2- 172.21.15.102:0/1649015188 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5aa009a620 0x7f5aa0096c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.178+0000 7f5aa546f640 1 -- 172.21.15.102:0/1649015188 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa009ad50 con 0x7f5aa0097170 2024-09-13T17:42:18.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.178+0000 7f5aa6471640 1 --2- 172.21.15.102:0/1649015188 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5aa012d180 0x7f5aa012d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:18.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.178+0000 7f5aa5c70640 1 --2- 172.21.15.102:0/1649015188 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5aa009a620 0x7f5aa0096c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:18.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.179+0000 7f5aa546f640 1 --2- 172.21.15.102:0/1649015188 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5aa0097170 0x7f5aa0097540 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5a90009940 tx=0x7f5a900091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6e9d02e6556084 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:18.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.179+0000 7f5aa4c6e640 1 -- 172.21.15.102:0/1649015188 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9002ad20 con 0x7f5aa0097170 2024-09-13T17:42:18.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.179+0000 7f5aa4c6e640 1 -- 172.21.15.102:0/1649015188 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9002ae80 con 0x7f5aa0097170 2024-09-13T17:42:18.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.179+0000 7f5aa4c6e640 1 -- 172.21.15.102:0/1649015188 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a90032670 con 0x7f5aa0097170 2024-09-13T17:42:18.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.179+0000 7f5aa7ed4640 1 -- 172.21.15.102:0/1649015188 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5aa0097170 msgr2=0x7f5aa0097540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:18.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.179+0000 7f5aa7ed4640 1 --2- 172.21.15.102:0/1649015188 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5aa0097170 0x7f5aa0097540 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5a90009940 tx=0x7f5a900091d0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.180+0000 7f5aa7ed4640 1 -- 172.21.15.102:0/1649015188 shutdown_connections 2024-09-13T17:42:18.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.180+0000 7f5aa7ed4640 1 --2- 172.21.15.102:0/1649015188 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5aa012d180 0x7f5aa012d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.180+0000 7f5aa7ed4640 1 --2- 172.21.15.102:0/1649015188 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5aa0097170 0x7f5aa0097540 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.180+0000 7f5aa7ed4640 1 --2- 172.21.15.102:0/1649015188 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5aa009a620 0x7f5aa0096c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.180+0000 7f5aa7ed4640 1 -- 172.21.15.102:0/1649015188 >> 172.21.15.102:0/1649015188 conn(0x7f5aa009e6b0 msgr2=0x7f5aa009eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:18.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.180+0000 7f5aa7ed4640 1 -- 172.21.15.102:0/1649015188 shutdown_connections 2024-09-13T17:42:18.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.180+0000 7f5aa7ed4640 1 -- 172.21.15.102:0/1649015188 wait complete. 2024-09-13T17:42:18.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.180+0000 7f5aa7ed4640 1 Processor -- start 2024-09-13T17:42:18.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.180+0000 7f5aa7ed4640 1 -- start start 2024-09-13T17:42:18.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.181+0000 7f5aa7ed4640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5aa0097170 0x7f5aa012c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:18.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.181+0000 7f5aa7ed4640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5aa009a620 0x7f5aa01186e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:18.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.181+0000 7f5aa7ed4640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5aa012d180 0x7f5aa0118c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:18.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.181+0000 7f5aa7ed4640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa011a620 con 0x7f5aa009a620 2024-09-13T17:42:18.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.181+0000 7f5aa7ed4640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa011bac0 con 0x7f5aa012d180 2024-09-13T17:42:18.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.181+0000 7f5aa7ed4640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa011bd10 con 0x7f5aa0097170 2024-09-13T17:42:18.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.181+0000 7f5aa5c70640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5aa0097170 0x7f5aa012c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:18.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.181+0000 7f5aa6471640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5aa012d180 0x7f5aa0118c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:18.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.181+0000 7f5aa546f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5aa009a620 0x7f5aa01186e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:18.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.181+0000 7f5aa6471640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5aa012d180 0x7f5aa0118c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35676/0 (socket says 172.21.15.102:35676) 2024-09-13T17:42:18.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.181+0000 7f5aa6471640 1 -- 172.21.15.102:0/4079384339 learned_addr learned my addr 172.21.15.102:0/4079384339 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:18.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.181+0000 7f5aa5c70640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5aa0097170 0x7f5aa012c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:46876/0 (socket says 172.21.15.102:46876) 2024-09-13T17:42:18.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.181+0000 7f5aa6471640 1 -- 172.21.15.102:0/4079384339 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5aa0097170 msgr2=0x7f5aa012c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:18.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.181+0000 7f5aa6471640 1 --2- 172.21.15.102:0/4079384339 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5aa0097170 0x7f5aa012c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.181+0000 7f5aa6471640 1 -- 172.21.15.102:0/4079384339 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5aa009a620 msgr2=0x7f5aa01186e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:18.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.181+0000 7f5aa6471640 1 --2- 172.21.15.102:0/4079384339 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5aa009a620 0x7f5aa01186e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.181+0000 7f5aa6471640 1 -- 172.21.15.102:0/4079384339 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a9c009390 con 0x7f5aa012d180 2024-09-13T17:42:18.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.181+0000 7f5aa6471640 1 --2- 172.21.15.102:0/4079384339 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5aa012d180 0x7f5aa0118c20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5a98009fc0 tx=0x7f5a9800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:18.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.181+0000 7f5a95ffb640 1 -- 172.21.15.102:0/4079384339 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9800ce70 con 0x7f5aa012d180 2024-09-13T17:42:18.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.182+0000 7f5a95ffb640 1 -- 172.21.15.102:0/4079384339 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9800f040 con 0x7f5aa012d180 2024-09-13T17:42:18.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.182+0000 7f5a95ffb640 1 -- 172.21.15.102:0/4079384339 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a980146d0 con 0x7f5aa012d180 2024-09-13T17:42:18.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.182+0000 7f5aa7ed4640 1 -- 172.21.15.102:0/4079384339 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a98009a20 con 0x7f5aa012d180 2024-09-13T17:42:18.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.182+0000 7f5aa7ed4640 1 -- 172.21.15.102:0/4079384339 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a90009450 con 0x7f5aa012d180 2024-09-13T17:42:18.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.182+0000 7f5a95ffb640 1 -- 172.21.15.102:0/4079384339 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f5a98003020 con 0x7f5aa012d180 2024-09-13T17:42:18.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.182+0000 7f5a95ffb640 1 -- 172.21.15.102:0/4079384339 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7f5a98042a60 con 0x7f5aa012d180 2024-09-13T17:42:18.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.183+0000 7f5aa7ed4640 1 -- 172.21.15.102:0/4079384339 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aa0064000 con 0x7f5aa012d180 2024-09-13T17:42:18.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.187+0000 7f5a95ffb640 1 -- 172.21.15.102:0/4079384339 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f5a98018080 con 0x7f5aa012d180 2024-09-13T17:42:18.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.340+0000 7f5aa7ed4640 1 -- 172.21.15.102:0/4079384339 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5aa006e5e0 con 0x7f5aa012d180 2024-09-13T17:42:18.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.340+0000 7f5a95ffb640 1 -- 172.21.15.102:0/4079384339 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5a98012d90 con 0x7f5aa012d180 2024-09-13T17:42:18.342 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-09-13T17:42:18.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.342+0000 7f5aa7ed4640 1 -- 172.21.15.102:0/4079384339 >> v1:172.21.15.102:6816/34937 conn(0x7f5a84031160 legacy=0x7f5a84033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:18.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.342+0000 7f5aa7ed4640 1 -- 172.21.15.102:0/4079384339 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5aa012d180 msgr2=0x7f5aa0118c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:18.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.342+0000 7f5aa7ed4640 1 --2- 172.21.15.102:0/4079384339 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5aa012d180 0x7f5aa0118c20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5a98009fc0 tx=0x7f5a9800b040 comp rx=0 tx=0).stop 2024-09-13T17:42:18.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.342+0000 7f5aa7ed4640 1 -- 172.21.15.102:0/4079384339 shutdown_connections 2024-09-13T17:42:18.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.342+0000 7f5aa7ed4640 1 --2- 172.21.15.102:0/4079384339 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5aa012d180 0x7f5aa0118c20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.342+0000 7f5aa7ed4640 1 --2- 172.21.15.102:0/4079384339 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5aa009a620 0x7f5aa01186e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.342+0000 7f5aa7ed4640 1 --2- 172.21.15.102:0/4079384339 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5aa0097170 0x7f5aa012c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.342+0000 7f5aa7ed4640 1 -- 172.21.15.102:0/4079384339 >> 172.21.15.102:0/4079384339 conn(0x7f5aa009e6b0 msgr2=0x7f5aa009d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:18.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.342+0000 7f5aa7ed4640 1 -- 172.21.15.102:0/4079384339 shutdown_connections 2024-09-13T17:42:18.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.342+0000 7f5aa7ed4640 1 -- 172.21.15.102:0/4079384339 wait complete. 2024-09-13T17:42:18.354 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:42:18.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.449+0000 7fa190f1d640 1 Processor -- start 2024-09-13T17:42:18.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.449+0000 7fa190f1d640 1 -- start start 2024-09-13T17:42:18.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.449+0000 7fa190f1d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa18c06db60 0x7fa18c12d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:18.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.449+0000 7fa190f1d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa18c12dc40 0x7fa18c09a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:18.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.450+0000 7fa190f1d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa18c12e050 0x7fa18c09ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:18.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.450+0000 7fa190f1d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa18c068810 con 0x7fa18c06db60 2024-09-13T17:42:18.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.450+0000 7fa190f1d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c06d360 con 0x7fa18c12dc40 2024-09-13T17:42:18.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.450+0000 7fa190f1d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c05e360 con 0x7fa18c12e050 2024-09-13T17:42:18.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.450+0000 7fa18a59c640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa18c06db60 0x7fa18c12d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:18.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.450+0000 7fa183fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa18c12dc40 0x7fa18c09a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:18.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.450+0000 7fa18a59c640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa18c06db60 0x7fa18c12d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37612/0 (socket says 172.21.15.102:37612) 2024-09-13T17:42:18.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.450+0000 7fa18ad9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa18c12e050 0x7fa18c09ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:18.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.450+0000 7fa18a59c640 1 -- 172.21.15.102:0/2503680651 learned_addr learned my addr 172.21.15.102:0/2503680651 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:18.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.450+0000 7fa183fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa18c12dc40 0x7fa18c09a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35680/0 (socket says 172.21.15.102:35680) 2024-09-13T17:42:18.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.450+0000 7fa18a59c640 1 -- 172.21.15.102:0/2503680651 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa18c12e050 msgr2=0x7fa18c09ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:18.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.450+0000 7fa18a59c640 1 --2- 172.21.15.102:0/2503680651 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa18c12e050 0x7fa18c09ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.450+0000 7fa18a59c640 1 -- 172.21.15.102:0/2503680651 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa18c12dc40 msgr2=0x7fa18c09a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:18.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.450+0000 7fa18a59c640 1 --2- 172.21.15.102:0/2503680651 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa18c12dc40 0x7fa18c09a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.450+0000 7fa18a59c640 1 -- 172.21.15.102:0/2503680651 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18c05df70 con 0x7fa18c06db60 2024-09-13T17:42:18.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.450+0000 7fa183fff640 1 --2- 172.21.15.102:0/2503680651 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa18c12dc40 0x7fa18c09a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:18.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.450+0000 7fa18ad9d640 1 --2- 172.21.15.102:0/2503680651 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa18c12e050 0x7fa18c09ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:18.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.451+0000 7fa18a59c640 1 --2- 172.21.15.102:0/2503680651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa18c06db60 0x7fa18c12d700 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa17c009910 tx=0x7fa17c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec810c1766814a06 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:18.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.451+0000 7fa189d9b640 1 -- 172.21.15.102:0/2503680651 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c02ac90 con 0x7fa18c06db60 2024-09-13T17:42:18.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.451+0000 7fa189d9b640 1 -- 172.21.15.102:0/2503680651 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17c02adf0 con 0x7fa18c06db60 2024-09-13T17:42:18.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.451+0000 7fa189d9b640 1 -- 172.21.15.102:0/2503680651 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c032670 con 0x7fa18c06db60 2024-09-13T17:42:18.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.451+0000 7fa190f1d640 1 -- 172.21.15.102:0/2503680651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa18c06db60 msgr2=0x7fa18c12d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:18.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.451+0000 7fa190f1d640 1 --2- 172.21.15.102:0/2503680651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa18c06db60 0x7fa18c12d700 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa17c009910 tx=0x7fa17c0091d0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.451+0000 7fa190f1d640 1 -- 172.21.15.102:0/2503680651 shutdown_connections 2024-09-13T17:42:18.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.451+0000 7fa190f1d640 1 --2- 172.21.15.102:0/2503680651 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa18c12e050 0x7fa18c09ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.451+0000 7fa190f1d640 1 --2- 172.21.15.102:0/2503680651 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa18c12dc40 0x7fa18c09a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.451+0000 7fa190f1d640 1 --2- 172.21.15.102:0/2503680651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa18c06db60 0x7fa18c12d700 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.451+0000 7fa190f1d640 1 -- 172.21.15.102:0/2503680651 >> 172.21.15.102:0/2503680651 conn(0x7fa18c09f960 msgr2=0x7fa18c09ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:18.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.451+0000 7fa190f1d640 1 -- 172.21.15.102:0/2503680651 shutdown_connections 2024-09-13T17:42:18.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.452+0000 7fa190f1d640 1 -- 172.21.15.102:0/2503680651 wait complete. 2024-09-13T17:42:18.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.452+0000 7fa190f1d640 1 Processor -- start 2024-09-13T17:42:18.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.452+0000 7fa190f1d640 1 -- start start 2024-09-13T17:42:18.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.453+0000 7fa190f1d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa18c06db60 0x7fa18c12a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:18.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.453+0000 7fa190f1d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa18c12dc40 0x7fa18c095aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:18.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.453+0000 7fa190f1d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa18c12e050 0x7fa18c06a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:18.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.453+0000 7fa190f1d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c129370 con 0x7fa18c12dc40 2024-09-13T17:42:18.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.453+0000 7fa190f1d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c06bf80 con 0x7fa18c06db60 2024-09-13T17:42:18.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.453+0000 7fa190f1d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c129d80 con 0x7fa18c12e050 2024-09-13T17:42:18.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.453+0000 7fa18a59c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa18c06db60 0x7fa18c12a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:18.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.453+0000 7fa18ad9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa18c12e050 0x7fa18c06a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:18.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.453+0000 7fa183fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa18c12dc40 0x7fa18c095aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:18.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.453+0000 7fa18a59c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa18c06db60 0x7fa18c12a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35684/0 (socket says 172.21.15.102:35684) 2024-09-13T17:42:18.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.453+0000 7fa18a59c640 1 -- 172.21.15.102:0/1418771574 learned_addr learned my addr 172.21.15.102:0/1418771574 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:18.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.453+0000 7fa183fff640 1 --2- 172.21.15.102:0/1418771574 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa18c12dc40 0x7fa18c095aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37616/0 (socket says 172.21.15.102:37616) 2024-09-13T17:42:18.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.453+0000 7fa18a59c640 1 -- 172.21.15.102:0/1418771574 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa18c12e050 msgr2=0x7fa18c06a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:18.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.453+0000 7fa18a59c640 1 --2- 172.21.15.102:0/1418771574 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa18c12e050 0x7fa18c06a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.453+0000 7fa18a59c640 1 -- 172.21.15.102:0/1418771574 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa18c12dc40 msgr2=0x7fa18c095aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:18.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.453+0000 7fa18a59c640 1 --2- 172.21.15.102:0/1418771574 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa18c12dc40 0x7fa18c095aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.453+0000 7fa18a59c640 1 -- 172.21.15.102:0/1418771574 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa174009390 con 0x7fa18c06db60 2024-09-13T17:42:18.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.453+0000 7fa183fff640 1 --2- 172.21.15.102:0/1418771574 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa18c12dc40 0x7fa18c095aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:18.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.453+0000 7fa18a59c640 1 --2- 172.21.15.102:0/1418771574 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa18c06db60 0x7fa18c12a820 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa17c02a5c0 tx=0x7fa17c032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:18.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.453+0000 7fa1827fc640 1 -- 172.21.15.102:0/1418771574 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c03d630 con 0x7fa18c06db60 2024-09-13T17:42:18.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.453+0000 7fa1827fc640 1 -- 172.21.15.102:0/1418771574 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17c03d7b0 con 0x7fa18c06db60 2024-09-13T17:42:18.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.453+0000 7fa1827fc640 1 -- 172.21.15.102:0/1418771574 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c03dbb0 con 0x7fa18c06db60 2024-09-13T17:42:18.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.453+0000 7fa190f1d640 1 -- 172.21.15.102:0/1418771574 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa184009a20 con 0x7fa18c06db60 2024-09-13T17:42:18.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.453+0000 7fa190f1d640 1 -- 172.21.15.102:0/1418771574 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa17c009450 con 0x7fa18c06db60 2024-09-13T17:42:18.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.454+0000 7fa1827fc640 1 -- 172.21.15.102:0/1418771574 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fa17c039040 con 0x7fa18c06db60 2024-09-13T17:42:18.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.454+0000 7fa1827fc640 1 -- 172.21.15.102:0/1418771574 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7fa17c061900 con 0x7fa18c06db60 2024-09-13T17:42:18.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.455+0000 7fa190f1d640 1 -- 172.21.15.102:0/1418771574 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa18c06cbc0 con 0x7fa18c06db60 2024-09-13T17:42:18.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.459+0000 7fa1827fc640 1 -- 172.21.15.102:0/1418771574 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fa17c0394f0 con 0x7fa18c06db60 2024-09-13T17:42:18.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.610+0000 7fa190f1d640 1 -- 172.21.15.102:0/1418771574 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa18c064000 con 0x7fa18c06db60 2024-09-13T17:42:18.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.610+0000 7fa1827fc640 1 -- 172.21.15.102:0/1418771574 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa17c037030 con 0x7fa18c06db60 2024-09-13T17:42:18.612 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-09-13T17:42:18.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.612+0000 7fa190f1d640 1 -- 172.21.15.102:0/1418771574 >> v1:172.21.15.102:6816/34937 conn(0x7fa16c031160 legacy=0x7fa16c033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:18.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.612+0000 7fa190f1d640 1 -- 172.21.15.102:0/1418771574 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa18c06db60 msgr2=0x7fa18c12a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:18.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.612+0000 7fa190f1d640 1 --2- 172.21.15.102:0/1418771574 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa18c06db60 0x7fa18c12a820 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa17c02a5c0 tx=0x7fa17c032000 comp rx=0 tx=0).stop 2024-09-13T17:42:18.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.612+0000 7fa190f1d640 1 -- 172.21.15.102:0/1418771574 shutdown_connections 2024-09-13T17:42:18.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.612+0000 7fa190f1d640 1 --2- 172.21.15.102:0/1418771574 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa18c12e050 0x7fa18c06a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.612+0000 7fa190f1d640 1 --2- 172.21.15.102:0/1418771574 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa18c12dc40 0x7fa18c095aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.612+0000 7fa190f1d640 1 --2- 172.21.15.102:0/1418771574 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa18c06db60 0x7fa18c12a820 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.612+0000 7fa190f1d640 1 -- 172.21.15.102:0/1418771574 >> 172.21.15.102:0/1418771574 conn(0x7fa18c09f960 msgr2=0x7fa18c12e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:18.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.612+0000 7fa190f1d640 1 -- 172.21.15.102:0/1418771574 shutdown_connections 2024-09-13T17:42:18.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.612+0000 7fa190f1d640 1 -- 172.21.15.102:0/1418771574 wait complete. 2024-09-13T17:42:18.625 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:42:18.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.716+0000 7f850ef00640 1 Processor -- start 2024-09-13T17:42:18.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.716+0000 7f850ef00640 1 -- start start 2024-09-13T17:42:18.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.716+0000 7f850ef00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f850809add0 0x7f850809b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:18.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.716+0000 7f850ef00640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f850812d2f0 0x7f850812d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:18.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.716+0000 7f850ef00640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f850812dc20 0x7f8508134630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:18.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.716+0000 7f850ef00640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f850806de30 con 0x7f850812d2f0 2024-09-13T17:42:18.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.716+0000 7f850ef00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850806c710 con 0x7f850809add0 2024-09-13T17:42:18.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.716+0000 7f850ef00640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f850805e2f0 con 0x7f850812dc20 2024-09-13T17:42:18.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.716+0000 7f84fffff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f850812d2f0 0x7f850812d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:18.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.716+0000 7f850d49d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f850812dc20 0x7f8508134630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:18.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.716+0000 7f850cc9c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f850809add0 0x7f850809b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:18.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.716+0000 7f84fffff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f850812d2f0 0x7f850812d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37622/0 (socket says 172.21.15.102:37622) 2024-09-13T17:42:18.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.716+0000 7f850d49d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f850812dc20 0x7f8508134630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:46916/0 (socket says 172.21.15.102:46916) 2024-09-13T17:42:18.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.716+0000 7f84fffff640 1 -- 172.21.15.102:0/65156122 learned_addr learned my addr 172.21.15.102:0/65156122 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:18.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.716+0000 7f850cc9c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f850809add0 0x7f850809b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35690/0 (socket says 172.21.15.102:35690) 2024-09-13T17:42:18.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.717+0000 7f84fffff640 1 -- 172.21.15.102:0/65156122 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f850812dc20 msgr2=0x7f8508134630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:18.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.717+0000 7f84fffff640 1 --2- 172.21.15.102:0/65156122 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f850812dc20 0x7f8508134630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.717+0000 7f84fffff640 1 -- 172.21.15.102:0/65156122 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f850809add0 msgr2=0x7f850809b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:18.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.717+0000 7f84fffff640 1 --2- 172.21.15.102:0/65156122 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f850809add0 0x7f850809b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.717+0000 7f84fffff640 1 -- 172.21.15.102:0/65156122 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850806d4b0 con 0x7f850812d2f0 2024-09-13T17:42:18.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.717+0000 7f850cc9c640 1 --2- 172.21.15.102:0/65156122 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f850809add0 0x7f850809b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:18.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.717+0000 7f850d49d640 1 --2- 172.21.15.102:0/65156122 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f850812dc20 0x7f8508134630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:18.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.717+0000 7f84fffff640 1 --2- 172.21.15.102:0/65156122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f850812d2f0 0x7f850812d6e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009940 tx=0x7f84f80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=419ddd76bff6f1a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:18.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.718+0000 7f84ff7fe640 1 -- 172.21.15.102:0/65156122 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f802ad20 con 0x7f850812d2f0 2024-09-13T17:42:18.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.718+0000 7f84ff7fe640 1 -- 172.21.15.102:0/65156122 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f802ae80 con 0x7f850812d2f0 2024-09-13T17:42:18.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.718+0000 7f84ff7fe640 1 -- 172.21.15.102:0/65156122 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f8032670 con 0x7f850812d2f0 2024-09-13T17:42:18.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.718+0000 7f850ef00640 1 -- 172.21.15.102:0/65156122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f850812d2f0 msgr2=0x7f850812d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:18.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.718+0000 7f850ef00640 1 --2- 172.21.15.102:0/65156122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f850812d2f0 0x7f850812d6e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009940 tx=0x7f84f80091d0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.718+0000 7f850ef00640 1 -- 172.21.15.102:0/65156122 shutdown_connections 2024-09-13T17:42:18.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.718+0000 7f850ef00640 1 --2- 172.21.15.102:0/65156122 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f850812dc20 0x7f8508134630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.718+0000 7f850ef00640 1 --2- 172.21.15.102:0/65156122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f850812d2f0 0x7f850812d6e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.718+0000 7f850ef00640 1 --2- 172.21.15.102:0/65156122 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f850809add0 0x7f850809b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.718+0000 7f850ef00640 1 -- 172.21.15.102:0/65156122 >> 172.21.15.102:0/65156122 conn(0x7f850809f9d0 msgr2=0x7f850809ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:18.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.718+0000 7f850ef00640 1 -- 172.21.15.102:0/65156122 shutdown_connections 2024-09-13T17:42:18.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.718+0000 7f850ef00640 1 -- 172.21.15.102:0/65156122 wait complete. 2024-09-13T17:42:18.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.719+0000 7f850ef00640 1 Processor -- start 2024-09-13T17:42:18.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.719+0000 7f850ef00640 1 -- start start 2024-09-13T17:42:18.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.720+0000 7f850ef00640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f850809add0 0x7f8508088cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:18.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.720+0000 7f850ef00640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f850812d2f0 0x7f8508129da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:18.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.720+0000 7f850ef00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f850812dc20 0x7f850812a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:18.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.720+0000 7f850ef00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850809d830 con 0x7f850812dc20 2024-09-13T17:42:18.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.720+0000 7f850ef00640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850812ccf0 con 0x7f850809add0 2024-09-13T17:42:18.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.720+0000 7f850ef00640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85080884f0 con 0x7f850812d2f0 2024-09-13T17:42:18.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.720+0000 7f850cc9c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f850809add0 0x7f8508088cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:18.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.720+0000 7f84fffff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f850812d2f0 0x7f8508129da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:18.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.720+0000 7f850cc9c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f850809add0 0x7f8508088cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35704/0 (socket says 172.21.15.102:35704) 2024-09-13T17:42:18.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.720+0000 7f850cc9c640 1 -- 172.21.15.102:0/2452285090 learned_addr learned my addr 172.21.15.102:0/2452285090 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:18.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.720+0000 7f850d49d640 1 --2- 172.21.15.102:0/2452285090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f850812dc20 0x7f850812a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:18.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.720+0000 7f850cc9c640 1 -- 172.21.15.102:0/2452285090 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f850812d2f0 msgr2=0x7f8508129da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:18.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.720+0000 7f850cc9c640 1 --2- 172.21.15.102:0/2452285090 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f850812d2f0 0x7f8508129da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.720+0000 7f850cc9c640 1 -- 172.21.15.102:0/2452285090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f850812dc20 msgr2=0x7f850812a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:18.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.720+0000 7f850cc9c640 1 --2- 172.21.15.102:0/2452285090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f850812dc20 0x7f850812a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.720+0000 7f850cc9c640 1 -- 172.21.15.102:0/2452285090 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8504009390 con 0x7f850809add0 2024-09-13T17:42:18.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.720+0000 7f84fffff640 1 --2- 172.21.15.102:0/2452285090 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f850812d2f0 0x7f8508129da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:18.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.720+0000 7f850d49d640 1 --2- 172.21.15.102:0/2452285090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f850812dc20 0x7f850812a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:18.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.720+0000 7f850cc9c640 1 --2- 172.21.15.102:0/2452285090 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f850809add0 0x7f8508088cd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f850400ec80 tx=0x7f85040090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:18.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.720+0000 7f84fcff9640 1 -- 172.21.15.102:0/2452285090 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850400eec0 con 0x7f850809add0 2024-09-13T17:42:18.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.720+0000 7f84fcff9640 1 -- 172.21.15.102:0/2452285090 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850400cb00 con 0x7f850809add0 2024-09-13T17:42:18.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.720+0000 7f850ef00640 1 -- 172.21.15.102:0/2452285090 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8500009a20 con 0x7f850809add0 2024-09-13T17:42:18.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.720+0000 7f84fcff9640 1 -- 172.21.15.102:0/2452285090 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850400ce80 con 0x7f850809add0 2024-09-13T17:42:18.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.720+0000 7f850ef00640 1 -- 172.21.15.102:0/2452285090 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f8009450 con 0x7f850809add0 2024-09-13T17:42:18.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.721+0000 7f84fcff9640 1 -- 172.21.15.102:0/2452285090 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f8504013680 con 0x7f850809add0 2024-09-13T17:42:18.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.721+0000 7f84fcff9640 1 -- 172.21.15.102:0/2452285090 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7f8504042830 con 0x7f850809add0 2024-09-13T17:42:18.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.722+0000 7f850ef00640 1 -- 172.21.15.102:0/2452285090 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f850812bad0 con 0x7f850809add0 2024-09-13T17:42:18.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.726+0000 7f84fcff9640 1 -- 172.21.15.102:0/2452285090 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f8504021020 con 0x7f850809add0 2024-09-13T17:42:18.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.908+0000 7f850ef00640 1 -- 172.21.15.102:0/2452285090 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8508064000 con 0x7f850809add0 2024-09-13T17:42:18.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.908+0000 7f84fcff9640 1 -- 172.21.15.102:0/2452285090 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2266 (secure 0 0 0) 0x7f850401bb90 con 0x7f850809add0 2024-09-13T17:42:18.910 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:42:18.910 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:42:06.611970+0000","modified":"2024-09-13T17:42:09.305093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4210":{"gid":4210,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6817/3443280887","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":3443280887}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-09-13T17:42:18.910 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 5 2024-09-13T17:42:18.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.910+0000 7f850ef00640 1 -- 172.21.15.102:0/2452285090 >> v1:172.21.15.102:6816/34937 conn(0x7f84f4031160 legacy=0x7f84f4033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:18.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.910+0000 7f850ef00640 1 -- 172.21.15.102:0/2452285090 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f850809add0 msgr2=0x7f8508088cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:18.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.910+0000 7f850ef00640 1 --2- 172.21.15.102:0/2452285090 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f850809add0 0x7f8508088cd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f850400ec80 tx=0x7f85040090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.910+0000 7f850ef00640 1 -- 172.21.15.102:0/2452285090 shutdown_connections 2024-09-13T17:42:18.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.910+0000 7f850ef00640 1 --2- 172.21.15.102:0/2452285090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f850812dc20 0x7f850812a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.910+0000 7f850ef00640 1 --2- 172.21.15.102:0/2452285090 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f850812d2f0 0x7f8508129da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.910+0000 7f850ef00640 1 --2- 172.21.15.102:0/2452285090 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f850809add0 0x7f8508088cd0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.910+0000 7f850ef00640 1 -- 172.21.15.102:0/2452285090 >> 172.21.15.102:0/2452285090 conn(0x7f850809f9d0 msgr2=0x7f8508098a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:18.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.910+0000 7f850ef00640 1 -- 172.21.15.102:0/2452285090 shutdown_connections 2024-09-13T17:42:18.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:18.910+0000 7f850ef00640 1 -- 172.21.15.102:0/2452285090 wait complete. 2024-09-13T17:42:18.922 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:19.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.014+0000 7f255db1d640 1 Processor -- start 2024-09-13T17:42:19.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.014+0000 7f255db1d640 1 -- start start 2024-09-13T17:42:19.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.014+0000 7f255db1d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f255806b140 0x7f25580998c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:19.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.014+0000 7f255db1d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f255809ac20 0x7f2558099e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:19.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.014+0000 7f255db1d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f255809a340 0x7f255812d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:19.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.014+0000 7f255db1d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f255806a680 con 0x7f255809a340 2024-09-13T17:42:19.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.014+0000 7f255db1d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f255806ee90 con 0x7f255809ac20 2024-09-13T17:42:19.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.014+0000 7f255db1d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f255806ac90 con 0x7f255806b140 2024-09-13T17:42:19.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.014+0000 7f25577fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f255806b140 0x7f25580998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:19.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.014+0000 7f2556ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f255809ac20 0x7f2558099e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:19.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.014+0000 7f25577fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f255806b140 0x7f25580998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:46922/0 (socket says 172.21.15.102:46922) 2024-09-13T17:42:19.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.014+0000 7f2556ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f255809ac20 0x7f2558099e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35708/0 (socket says 172.21.15.102:35708) 2024-09-13T17:42:19.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.015+0000 7f25577fe640 1 -- 172.21.15.102:0/636589697 learned_addr learned my addr 172.21.15.102:0/636589697 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:19.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.014+0000 7f2557fff640 1 --2- 172.21.15.102:0/636589697 >> v2:172.21.15.102:3300/0 conn(0x7f255809a340 0x7f255812d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:19.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.015+0000 7f25577fe640 1 -- 172.21.15.102:0/636589697 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f255809ac20 msgr2=0x7f2558099e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:19.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.015+0000 7f25577fe640 1 --2- 172.21.15.102:0/636589697 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f255809ac20 0x7f2558099e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.015+0000 7f25577fe640 1 -- 172.21.15.102:0/636589697 >> v2:172.21.15.102:3300/0 conn(0x7f255809a340 msgr2=0x7f255812d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:19.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.015+0000 7f25577fe640 1 --2- 172.21.15.102:0/636589697 >> v2:172.21.15.102:3300/0 conn(0x7f255809a340 0x7f255812d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.015+0000 7f25577fe640 1 -- 172.21.15.102:0/636589697 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2558099510 con 0x7f255806b140 2024-09-13T17:42:19.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.015+0000 7f2556ffd640 1 --2- 172.21.15.102:0/636589697 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f255809ac20 0x7f2558099e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:19.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.015+0000 7f25577fe640 1 --2- 172.21.15.102:0/636589697 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f255806b140 0x7f25580998c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2540009910 tx=0x7f25400091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0a5739c99ff32b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:19.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.016+0000 7f25567fc640 1 -- 172.21.15.102:0/636589697 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254002ac80 con 0x7f255806b140 2024-09-13T17:42:19.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.016+0000 7f25567fc640 1 -- 172.21.15.102:0/636589697 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f254002ade0 con 0x7f255806b140 2024-09-13T17:42:19.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.016+0000 7f25567fc640 1 -- 172.21.15.102:0/636589697 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2540032670 con 0x7f255806b140 2024-09-13T17:42:19.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.016+0000 7f255db1d640 1 -- 172.21.15.102:0/636589697 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f255806b140 msgr2=0x7f25580998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:19.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.016+0000 7f255db1d640 1 --2- 172.21.15.102:0/636589697 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f255806b140 0x7f25580998c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2540009910 tx=0x7f25400091d0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.016+0000 7f255db1d640 1 -- 172.21.15.102:0/636589697 shutdown_connections 2024-09-13T17:42:19.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.016+0000 7f255db1d640 1 --2- 172.21.15.102:0/636589697 >> v2:172.21.15.102:3300/0 conn(0x7f255809a340 0x7f255812d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.016+0000 7f255db1d640 1 --2- 172.21.15.102:0/636589697 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f255809ac20 0x7f2558099e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.016+0000 7f255db1d640 1 --2- 172.21.15.102:0/636589697 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f255806b140 0x7f25580998c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.016+0000 7f255db1d640 1 -- 172.21.15.102:0/636589697 >> 172.21.15.102:0/636589697 conn(0x7f255809e820 msgr2=0x7f255809ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:19.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.016+0000 7f255db1d640 1 -- 172.21.15.102:0/636589697 shutdown_connections 2024-09-13T17:42:19.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.016+0000 7f255db1d640 1 -- 172.21.15.102:0/636589697 wait complete. 2024-09-13T17:42:19.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.017+0000 7f255db1d640 1 Processor -- start 2024-09-13T17:42:19.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.017+0000 7f255db1d640 1 -- start start 2024-09-13T17:42:19.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.018+0000 7f255db1d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f255806b140 0x7f255811ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:19.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.018+0000 7f255db1d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f255809a340 0x7f255811bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:19.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.018+0000 7f255db1d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f255809ac20 0x7f255811c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:19.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.018+0000 7f255db1d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255811da90 con 0x7f255809ac20 2024-09-13T17:42:19.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.018+0000 7f255db1d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f255811d580 con 0x7f255806b140 2024-09-13T17:42:19.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.018+0000 7f255db1d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f255811eef0 con 0x7f255809a340 2024-09-13T17:42:19.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.018+0000 7f25577fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f255806b140 0x7f255811ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:19.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.018+0000 7f2556ffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f255809a340 0x7f255811bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:19.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.018+0000 7f25577fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f255806b140 0x7f255811ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35716/0 (socket says 172.21.15.102:35716) 2024-09-13T17:42:19.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.018+0000 7f25577fe640 1 -- 172.21.15.102:0/1709536647 learned_addr learned my addr 172.21.15.102:0/1709536647 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:19.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.018+0000 7f2557fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f255809ac20 0x7f255811c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:19.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.018+0000 7f25577fe640 1 -- 172.21.15.102:0/1709536647 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f255809a340 msgr2=0x7f255811bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:19.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.018+0000 7f25577fe640 1 --2- 172.21.15.102:0/1709536647 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f255809a340 0x7f255811bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.018+0000 7f25577fe640 1 -- 172.21.15.102:0/1709536647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f255809ac20 msgr2=0x7f255811c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:19.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.018+0000 7f25577fe640 1 --2- 172.21.15.102:0/1709536647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f255809ac20 0x7f255811c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.018+0000 7f25577fe640 1 -- 172.21.15.102:0/1709536647 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2548009390 con 0x7f255806b140 2024-09-13T17:42:19.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.018+0000 7f2556ffd640 1 --2- 172.21.15.102:0/1709536647 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f255809a340 0x7f255811bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:19.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.018+0000 7f2557fff640 1 --2- 172.21.15.102:0/1709536647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f255809ac20 0x7f255811c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:19.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.018+0000 7f25577fe640 1 --2- 172.21.15.102:0/1709536647 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f255806b140 0x7f255811ba50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2540009970 tx=0x7f2540032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:19.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.018+0000 7f254f7fe640 1 -- 172.21.15.102:0/1709536647 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254003d630 con 0x7f255806b140 2024-09-13T17:42:19.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.018+0000 7f254f7fe640 1 -- 172.21.15.102:0/1709536647 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f254003d7b0 con 0x7f255806b140 2024-09-13T17:42:19.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.018+0000 7f255db1d640 1 -- 172.21.15.102:0/1709536647 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2540009450 con 0x7f255806b140 2024-09-13T17:42:19.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.018+0000 7f254f7fe640 1 -- 172.21.15.102:0/1709536647 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254003dbb0 con 0x7f255806b140 2024-09-13T17:42:19.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.018+0000 7f255db1d640 1 -- 172.21.15.102:0/1709536647 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f255811f260 con 0x7f255806b140 2024-09-13T17:42:19.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.019+0000 7f254f7fe640 1 -- 172.21.15.102:0/1709536647 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f2540039040 con 0x7f255806b140 2024-09-13T17:42:19.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.019+0000 7f254f7fe640 1 -- 172.21.15.102:0/1709536647 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7f2540061a50 con 0x7f255806b140 2024-09-13T17:42:19.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.019+0000 7f255db1d640 1 -- 172.21.15.102:0/1709536647 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2558064000 con 0x7f255806b140 2024-09-13T17:42:19.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.024+0000 7f254f7fe640 1 -- 172.21.15.102:0/1709536647 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f2540042080 con 0x7f255806b140 2024-09-13T17:42:19.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.175+0000 7f255db1d640 1 -- 172.21.15.102:0/1709536647 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2558069dd0 con 0x7f255806b140 2024-09-13T17:42:19.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.175+0000 7f254f7fe640 1 -- 172.21.15.102:0/1709536647 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+11375 (secure 0 0 0) 0x7f2540037030 con 0x7f255806b140 2024-09-13T17:42:19.177 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:42:19.178 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":20,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:42:07.375548+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:04.672265+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:05.583134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:19.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.177+0000 7f255db1d640 1 -- 172.21.15.102:0/1709536647 >> v1:172.21.15.102:6816/34937 conn(0x7f2538031160 legacy=0x7f2538033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:19.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.177+0000 7f255db1d640 1 -- 172.21.15.102:0/1709536647 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f255806b140 msgr2=0x7f255811ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:19.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.177+0000 7f255db1d640 1 --2- 172.21.15.102:0/1709536647 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f255806b140 0x7f255811ba50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2540009970 tx=0x7f2540032000 comp rx=0 tx=0).stop 2024-09-13T17:42:19.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.177+0000 7f255db1d640 1 -- 172.21.15.102:0/1709536647 shutdown_connections 2024-09-13T17:42:19.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.177+0000 7f255db1d640 1 --2- 172.21.15.102:0/1709536647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f255809ac20 0x7f255811c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.177+0000 7f255db1d640 1 --2- 172.21.15.102:0/1709536647 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f255809a340 0x7f255811bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.177+0000 7f255db1d640 1 --2- 172.21.15.102:0/1709536647 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f255806b140 0x7f255811ba50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.177+0000 7f255db1d640 1 -- 172.21.15.102:0/1709536647 >> 172.21.15.102:0/1709536647 conn(0x7f255809e820 msgr2=0x7f255806cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:19.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.178+0000 7f255db1d640 1 -- 172.21.15.102:0/1709536647 shutdown_connections 2024-09-13T17:42:19.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.178+0000 7f255db1d640 1 -- 172.21.15.102:0/1709536647 wait complete. 2024-09-13T17:42:19.191 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:42:19.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.282+0000 7fca4a754640 1 Processor -- start 2024-09-13T17:42:19.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.283+0000 7fca4a754640 1 -- start start 2024-09-13T17:42:19.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.283+0000 7fca4a754640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fca44068920 0x7fca4409d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:19.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.283+0000 7fca4a754640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fca4409dc50 0x7fca4409e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:19.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.283+0000 7fca4a754640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca440695a0 con 0x7fca4412f310 2024-09-13T17:42:19.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.283+0000 7fca4a754640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca44068cf0 con 0x7fca4409dc50 2024-09-13T17:42:19.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.283+0000 7fca4a754640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4409f090 con 0x7fca44068920 2024-09-13T17:42:19.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.283+0000 7fca437fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fca4409dc50 0x7fca4409e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:19.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.283+0000 7fca43fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fca44068920 0x7fca4409d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:19.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.283+0000 7fca48cf1640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fca4412f310 0x7fca4412f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:60960/0 (socket says 172.21.15.102:60960) 2024-09-13T17:42:19.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.283+0000 7fca48cf1640 1 -- 172.21.15.102:0/2797269833 learned_addr learned my addr 172.21.15.102:0/2797269833 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:42:19.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.283+0000 7fca437fe640 1 -- 172.21.15.102:0/2797269833 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fca44068920 msgr2=0x7fca4409d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:19.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.283+0000 7fca437fe640 1 --2- 172.21.15.102:0/2797269833 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fca44068920 0x7fca4409d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.283+0000 7fca437fe640 1 -- 172.21.15.102:0/2797269833 >> v1:172.21.15.102:6789/0 conn(0x7fca4412f310 legacy=0x7fca4412f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:19.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.283+0000 7fca437fe640 1 -- 172.21.15.102:0/2797269833 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4409cce0 con 0x7fca4409dc50 2024-09-13T17:42:19.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.283+0000 7fca42ffd640 1 -- 172.21.15.102:0/2797269833 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 94948131 0 0) 0x7fca440695a0 con 0x7fca4412f310 2024-09-13T17:42:19.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.283+0000 7fca43fff640 1 --2- 172.21.15.102:0/2797269833 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fca44068920 0x7fca4409d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:19.430 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.284+0000 7fca437fe640 1 --2- 172.21.15.102:0/2797269833 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fca4409dc50 0x7fca4409e020 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009850 tx=0x7fca2c0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c06f089873218c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:19.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.284+0000 7fca42ffd640 1 -- 172.21.15.102:0/2797269833 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c02ad00 con 0x7fca4409dc50 2024-09-13T17:42:19.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.284+0000 7fca42ffd640 1 -- 172.21.15.102:0/2797269833 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2c02ae60 con 0x7fca4409dc50 2024-09-13T17:42:19.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.284+0000 7fca42ffd640 1 -- 172.21.15.102:0/2797269833 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c032670 con 0x7fca4409dc50 2024-09-13T17:42:19.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.285+0000 7fca4a754640 1 -- 172.21.15.102:0/2797269833 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fca4409dc50 msgr2=0x7fca4409e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:19.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.285+0000 7fca4a754640 1 --2- 172.21.15.102:0/2797269833 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fca4409dc50 0x7fca4409e020 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009850 tx=0x7fca2c0090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.285+0000 7fca4a754640 1 -- 172.21.15.102:0/2797269833 shutdown_connections 2024-09-13T17:42:19.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.285+0000 7fca4a754640 1 --2- 172.21.15.102:0/2797269833 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fca4409dc50 0x7fca4409e020 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.285+0000 7fca4a754640 1 --2- 172.21.15.102:0/2797269833 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fca44068920 0x7fca4409d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.285+0000 7fca4a754640 1 -- 172.21.15.102:0/2797269833 >> 172.21.15.102:0/2797269833 conn(0x7fca4406f320 msgr2=0x7fca440a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:19.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.285+0000 7fca4a754640 1 -- 172.21.15.102:0/2797269833 shutdown_connections 2024-09-13T17:42:19.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.285+0000 7fca4a754640 1 -- 172.21.15.102:0/2797269833 wait complete. 2024-09-13T17:42:19.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.286+0000 7fca4a754640 1 Processor -- start 2024-09-13T17:42:19.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.286+0000 7fca4a754640 1 -- start start 2024-09-13T17:42:19.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.286+0000 7fca4a754640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fca44068920 0x7fca4411ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:19.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.286+0000 7fca4a754640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fca4409dc50 0x7fca4411b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:19.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.286+0000 7fca4a754640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fca4412f310 0x7fca4411b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:19.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.286+0000 7fca4a754640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4411cc80 con 0x7fca44068920 2024-09-13T17:42:19.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.286+0000 7fca4a754640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4411db50 con 0x7fca4409dc50 2024-09-13T17:42:19.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.286+0000 7fca4a754640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca4409fe50 con 0x7fca4412f310 2024-09-13T17:42:19.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.286+0000 7fca43fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fca44068920 0x7fca4411ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:19.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.286+0000 7fca43fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fca44068920 0x7fca4411ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37674/0 (socket says 172.21.15.102:37674) 2024-09-13T17:42:19.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.286+0000 7fca43fff640 1 -- 172.21.15.102:0/351319362 learned_addr learned my addr 172.21.15.102:0/351319362 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:19.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.286+0000 7fca437fe640 1 --2- 172.21.15.102:0/351319362 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fca4409dc50 0x7fca4411b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:19.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.286+0000 7fca48cf1640 1 --2- 172.21.15.102:0/351319362 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fca4412f310 0x7fca4411b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:19.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.287+0000 7fca43fff640 1 -- 172.21.15.102:0/351319362 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fca4412f310 msgr2=0x7fca4411b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:19.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.287+0000 7fca43fff640 1 --2- 172.21.15.102:0/351319362 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fca4412f310 0x7fca4411b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.287+0000 7fca43fff640 1 -- 172.21.15.102:0/351319362 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fca4409dc50 msgr2=0x7fca4411b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:19.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.287+0000 7fca43fff640 1 --2- 172.21.15.102:0/351319362 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fca4409dc50 0x7fca4411b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.287+0000 7fca43fff640 1 -- 172.21.15.102:0/351319362 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca34009390 con 0x7fca44068920 2024-09-13T17:42:19.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.287+0000 7fca43fff640 1 --2- 172.21.15.102:0/351319362 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fca44068920 0x7fca4411ada0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fca3400ec80 tx=0x7fca340090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:19.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.287+0000 7fca3bfff640 1 -- 172.21.15.102:0/351319362 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3400eec0 con 0x7fca44068920 2024-09-13T17:42:19.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.287+0000 7fca3bfff640 1 -- 172.21.15.102:0/351319362 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3400cbc0 con 0x7fca44068920 2024-09-13T17:42:19.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.287+0000 7fca4a754640 1 -- 172.21.15.102:0/351319362 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca2c009390 con 0x7fca44068920 2024-09-13T17:42:19.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.287+0000 7fca3bfff640 1 -- 172.21.15.102:0/351319362 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca34013470 con 0x7fca44068920 2024-09-13T17:42:19.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.287+0000 7fca4a754640 1 -- 172.21.15.102:0/351319362 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca4411e0f0 con 0x7fca44068920 2024-09-13T17:42:19.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.288+0000 7fca3bfff640 1 -- 172.21.15.102:0/351319362 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fca340137e0 con 0x7fca44068920 2024-09-13T17:42:19.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.288+0000 7fca3bfff640 1 -- 172.21.15.102:0/351319362 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7fca34042450 con 0x7fca44068920 2024-09-13T17:42:19.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.288+0000 7fca4a754640 1 -- 172.21.15.102:0/351319362 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca44064000 con 0x7fca44068920 2024-09-13T17:42:19.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.292+0000 7fca3bfff640 1 -- 172.21.15.102:0/351319362 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fca34023ad0 con 0x7fca44068920 2024-09-13T17:42:19.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.481+0000 7fca4a754640 1 -- 172.21.15.102:0/351319362 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca4409c7d0 con 0x7fca44068920 2024-09-13T17:42:19.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.481+0000 7fca3bfff640 1 -- 172.21.15.102:0/351319362 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2266 (secure 0 0 0) 0x7fca3402b090 con 0x7fca44068920 2024-09-13T17:42:19.483 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:42:19.483 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:42:06.611970+0000","modified":"2024-09-13T17:42:09.305093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4210":{"gid":4210,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6817/3443280887","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":3443280887}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-09-13T17:42:19.483 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 5 2024-09-13T17:42:19.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.483+0000 7fca4a754640 1 -- 172.21.15.102:0/351319362 >> v1:172.21.15.102:6816/34937 conn(0x7fca24031160 legacy=0x7fca24033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:19.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.483+0000 7fca4a754640 1 -- 172.21.15.102:0/351319362 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fca44068920 msgr2=0x7fca4411ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:19.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.483+0000 7fca4a754640 1 --2- 172.21.15.102:0/351319362 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fca44068920 0x7fca4411ada0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fca3400ec80 tx=0x7fca340090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.483+0000 7fca4a754640 1 -- 172.21.15.102:0/351319362 shutdown_connections 2024-09-13T17:42:19.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.483+0000 7fca4a754640 1 --2- 172.21.15.102:0/351319362 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fca4412f310 0x7fca4411b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.483+0000 7fca4a754640 1 --2- 172.21.15.102:0/351319362 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fca4409dc50 0x7fca4411b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.483+0000 7fca4a754640 1 --2- 172.21.15.102:0/351319362 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fca44068920 0x7fca4411ada0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.483+0000 7fca4a754640 1 -- 172.21.15.102:0/351319362 >> 172.21.15.102:0/351319362 conn(0x7fca4406f320 msgr2=0x7fca440a1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:19.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.483+0000 7fca4a754640 1 -- 172.21.15.102:0/351319362 shutdown_connections 2024-09-13T17:42:19.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.483+0000 7fca4a754640 1 -- 172.21.15.102:0/351319362 wait complete. 2024-09-13T17:42:19.596 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T17:42:06.611970+0000', 'modified': '2024-09-13T17:42:09.305093+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4210}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4210': {'gid': 4210, 'name': 'a', 'rank': 0, 'incarnation': 3, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.102:6817/3443280887', 'addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.102:6817', 'nonce': 3443280887}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-09-13T17:42:19.596 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T17:42:19.596 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-13T17:42:19.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.689+0000 7fba0c64f640 1 Processor -- start 2024-09-13T17:42:19.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.689+0000 7fba0c64f640 1 -- start start 2024-09-13T17:42:19.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.689+0000 7fba0c64f640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fba04098ef0 0x7fba040992c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:19.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.689+0000 7fba0c64f640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fba04067c30 0x7fba04137760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:19.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.689+0000 7fba0c64f640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba0409a0e0 con 0x7fba04067130 2024-09-13T17:42:19.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.689+0000 7fba0c64f640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba04097600 con 0x7fba04098ef0 2024-09-13T17:42:19.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.689+0000 7fba0c64f640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fba04096ea0 con 0x7fba04067c30 2024-09-13T17:42:19.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.689+0000 7fba0a3eb640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fba04098ef0 0x7fba040992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:19.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.689+0000 7fba0abec640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fba04067c30 0x7fba04137760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:19.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.690+0000 7fba0abec640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fba04067c30 0x7fba04137760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:46948/0 (socket says 172.21.15.102:46948) 2024-09-13T17:42:19.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.689+0000 7fba0a3eb640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fba04098ef0 0x7fba040992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37688/0 (socket says 172.21.15.102:37688) 2024-09-13T17:42:19.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.690+0000 7fba0abec640 1 -- 172.21.15.102:0/1378928543 learned_addr learned my addr 172.21.15.102:0/1378928543 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:19.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.689+0000 7fba09bea640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fba04067130 0x7fba04067520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:60974/0 (socket says 172.21.15.102:60974) 2024-09-13T17:42:19.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.690+0000 7fba0abec640 1 -- 172.21.15.102:0/1378928543 >> v2:172.21.15.102:3300/0 conn(0x7fba04098ef0 msgr2=0x7fba040992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:19.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.690+0000 7fba0abec640 1 --2- 172.21.15.102:0/1378928543 >> v2:172.21.15.102:3300/0 conn(0x7fba04098ef0 0x7fba040992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.690+0000 7fba0abec640 1 -- 172.21.15.102:0/1378928543 >> v1:172.21.15.102:6789/0 conn(0x7fba04067130 legacy=0x7fba04067520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:19.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.690+0000 7fba0abec640 1 -- 172.21.15.102:0/1378928543 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba04097cc0 con 0x7fba04067c30 2024-09-13T17:42:19.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.690+0000 7fba093e9640 1 -- 172.21.15.102:0/1378928543 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 818262112 0 0) 0x7fba0409a0e0 con 0x7fba04067130 2024-09-13T17:42:19.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.690+0000 7fba0a3eb640 1 --2- 172.21.15.102:0/1378928543 >> v2:172.21.15.102:3300/0 conn(0x7fba04098ef0 0x7fba040992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:19.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.691+0000 7fba0abec640 1 --2- 172.21.15.102:0/1378928543 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fba04067c30 0x7fba04137760 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc009f20 tx=0x7fb9fc009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24bbe9ddc6fe9b81 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:19.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.691+0000 7fba093e9640 1 -- 172.21.15.102:0/1378928543 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc031400 con 0x7fba04067c30 2024-09-13T17:42:19.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.691+0000 7fba093e9640 1 -- 172.21.15.102:0/1378928543 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9fc02d050 con 0x7fba04067c30 2024-09-13T17:42:19.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.691+0000 7fba093e9640 1 -- 172.21.15.102:0/1378928543 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc031790 con 0x7fba04067c30 2024-09-13T17:42:19.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.691+0000 7fba0c64f640 1 -- 172.21.15.102:0/1378928543 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fba04067c30 msgr2=0x7fba04137760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:19.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.691+0000 7fba0c64f640 1 --2- 172.21.15.102:0/1378928543 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fba04067c30 0x7fba04137760 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc009f20 tx=0x7fb9fc009840 comp rx=0 tx=0).stop 2024-09-13T17:42:19.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.691+0000 7fba0c64f640 1 -- 172.21.15.102:0/1378928543 shutdown_connections 2024-09-13T17:42:19.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.691+0000 7fba0c64f640 1 --2- 172.21.15.102:0/1378928543 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fba04067c30 0x7fba04137760 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.691+0000 7fba0c64f640 1 --2- 172.21.15.102:0/1378928543 >> v2:172.21.15.102:3300/0 conn(0x7fba04098ef0 0x7fba040992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.691+0000 7fba0c64f640 1 -- 172.21.15.102:0/1378928543 >> 172.21.15.102:0/1378928543 conn(0x7fba0405db50 msgr2=0x7fba0405df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:19.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.691+0000 7fba0c64f640 1 -- 172.21.15.102:0/1378928543 shutdown_connections 2024-09-13T17:42:19.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.691+0000 7fba0c64f640 1 -- 172.21.15.102:0/1378928543 wait complete. 2024-09-13T17:42:19.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.692+0000 7fba0c64f640 1 Processor -- start 2024-09-13T17:42:19.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.692+0000 7fba0c64f640 1 -- start start 2024-09-13T17:42:19.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.693+0000 7fba0c64f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba04067130 0x7fba0413e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:19.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.693+0000 7fba0c64f640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fba04067c30 0x7fba0413eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:19.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.693+0000 7fba0c64f640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fba04098ef0 0x7fba0413f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:19.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.693+0000 7fba0c64f640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba041223c0 con 0x7fba04067130 2024-09-13T17:42:19.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.693+0000 7fba0c64f640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba0409b460 con 0x7fba04067c30 2024-09-13T17:42:19.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.693+0000 7fba0c64f640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fba04121e20 con 0x7fba04098ef0 2024-09-13T17:42:19.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.693+0000 7fba0a3eb640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba04067130 0x7fba0413e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:19.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.693+0000 7fba0abec640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fba04098ef0 0x7fba0413f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:19.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.693+0000 7fba0a3eb640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba04067130 0x7fba0413e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37692/0 (socket says 172.21.15.102:37692) 2024-09-13T17:42:19.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.693+0000 7fba09bea640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fba04067c30 0x7fba0413eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:19.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.693+0000 7fba0a3eb640 1 -- 172.21.15.102:0/1611958714 learned_addr learned my addr 172.21.15.102:0/1611958714 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:19.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.693+0000 7fba0a3eb640 1 -- 172.21.15.102:0/1611958714 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fba04098ef0 msgr2=0x7fba0413f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:19.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.693+0000 7fba0a3eb640 1 --2- 172.21.15.102:0/1611958714 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fba04098ef0 0x7fba0413f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.693+0000 7fba0a3eb640 1 -- 172.21.15.102:0/1611958714 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fba04067c30 msgr2=0x7fba0413eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:19.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.693+0000 7fba0a3eb640 1 --2- 172.21.15.102:0/1611958714 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fba04067c30 0x7fba0413eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.693+0000 7fba0a3eb640 1 -- 172.21.15.102:0/1611958714 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f8009390 con 0x7fba04067130 2024-09-13T17:42:19.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.693+0000 7fba0abec640 1 --2- 172.21.15.102:0/1611958714 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fba04098ef0 0x7fba0413f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:19.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.693+0000 7fba09bea640 1 --2- 172.21.15.102:0/1611958714 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fba04067c30 0x7fba0413eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:19.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.693+0000 7fba0a3eb640 1 --2- 172.21.15.102:0/1611958714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba04067130 0x7fba0413e760 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800ec80 tx=0x7fb9f80090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:19.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.693+0000 7fb9f67fc640 1 -- 172.21.15.102:0/1611958714 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f800eec0 con 0x7fba04067130 2024-09-13T17:42:19.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.693+0000 7fb9f67fc640 1 -- 172.21.15.102:0/1611958714 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f800cbc0 con 0x7fba04067130 2024-09-13T17:42:19.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.693+0000 7fb9f67fc640 1 -- 172.21.15.102:0/1611958714 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8013470 con 0x7fba04067130 2024-09-13T17:42:19.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.693+0000 7fba0c64f640 1 -- 172.21.15.102:0/1611958714 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9fc009ae0 con 0x7fba04067130 2024-09-13T17:42:19.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.693+0000 7fba0c64f640 1 -- 172.21.15.102:0/1611958714 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba04120520 con 0x7fba04067130 2024-09-13T17:42:19.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.694+0000 7fb9f67fc640 1 -- 172.21.15.102:0/1611958714 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fb9f80137e0 con 0x7fba04067130 2024-09-13T17:42:19.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.694+0000 7fb9f67fc640 1 -- 172.21.15.102:0/1611958714 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7fb9f8042480 con 0x7fba04067130 2024-09-13T17:42:19.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.696+0000 7fba0ebc6740 1 Processor -- start 2024-09-13T17:42:19.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.696+0000 7fba0ebc6740 1 -- start start 2024-09-13T17:42:19.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.696+0000 7fba0ebc6740 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562377699960 0x56237767d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:19.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.696+0000 7fba0ebc6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562377699d30 0x56237769a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:19.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.696+0000 7fba0ebc6740 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5623776a6350 0x5623776a8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:19.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.696+0000 7fba0ebc6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623775e8e70 con 0x562377699d30 2024-09-13T17:42:19.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.696+0000 7fba0ebc6740 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x5623775e6600 con 0x562377699960 2024-09-13T17:42:19.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.696+0000 7fba0ebc6740 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x5623775e22e0 con 0x5623776a6350 2024-09-13T17:42:19.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.696+0000 7fba0a3eb640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562377699960 0x56237767d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:19.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.696+0000 7fba09bea640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562377699d30 0x56237769a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:19.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.696+0000 7fba0a3eb640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562377699960 0x56237767d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35748/0 (socket says 172.21.15.102:35748) 2024-09-13T17:42:19.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.696+0000 7fba09bea640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562377699d30 0x56237769a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37700/0 (socket says 172.21.15.102:37700) 2024-09-13T17:42:19.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.696+0000 7fba0abec640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5623776a6350 0x5623776a8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:19.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.696+0000 7fba0a3eb640 1 -- 172.21.15.102:0/2429758403 learned_addr learned my addr 172.21.15.102:0/2429758403 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:19.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.697+0000 7fba0a3eb640 1 -- 172.21.15.102:0/2429758403 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5623776a6350 msgr2=0x5623776a8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:19.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.697+0000 7fba0a3eb640 1 --2- 172.21.15.102:0/2429758403 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5623776a6350 0x5623776a8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.697+0000 7fba0a3eb640 1 -- 172.21.15.102:0/2429758403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562377699d30 msgr2=0x56237769a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:19.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.697+0000 7fba0a3eb640 1 --2- 172.21.15.102:0/2429758403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562377699d30 0x56237769a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.697+0000 7fba0a3eb640 1 -- 172.21.15.102:0/2429758403 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56237730a9e0 con 0x562377699960 2024-09-13T17:42:19.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.697+0000 7fba0abec640 1 --2- 172.21.15.102:0/2429758403 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5623776a6350 0x5623776a8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:19.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.697+0000 7fba09bea640 1 --2- 172.21.15.102:0/2429758403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562377699d30 0x56237769a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:19.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.697+0000 7fba0a3eb640 1 --2- 172.21.15.102:0/2429758403 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562377699960 0x56237767d6e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb9f80098b0 tx=0x7fb9f8017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c84a03f5456ee1c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:19.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.697+0000 7fb9e77fe640 1 -- 172.21.15.102:0/2429758403 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8023d00 con 0x562377699960 2024-09-13T17:42:19.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.697+0000 7fb9e77fe640 1 -- 172.21.15.102:0/2429758403 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f8023e60 con 0x562377699960 2024-09-13T17:42:19.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.697+0000 7fb9e77fe640 1 -- 172.21.15.102:0/2429758403 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f801b3b0 con 0x562377699960 2024-09-13T17:42:19.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.698+0000 7fba0ebc6740 1 -- 172.21.15.102:0/2429758403 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562377699960 msgr2=0x56237767d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:19.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.698+0000 7fba0ebc6740 1 --2- 172.21.15.102:0/2429758403 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562377699960 0x56237767d6e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb9f80098b0 tx=0x7fb9f8017000 comp rx=0 tx=0).stop 2024-09-13T17:42:19.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.698+0000 7fba0ebc6740 1 -- 172.21.15.102:0/2429758403 shutdown_connections 2024-09-13T17:42:19.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.698+0000 7fba0ebc6740 1 --2- 172.21.15.102:0/2429758403 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5623776a6350 0x5623776a8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.698+0000 7fba0ebc6740 1 --2- 172.21.15.102:0/2429758403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562377699d30 0x56237769a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.698+0000 7fba0ebc6740 1 --2- 172.21.15.102:0/2429758403 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562377699960 0x56237767d6e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.698+0000 7fba0ebc6740 1 -- 172.21.15.102:0/2429758403 >> 172.21.15.102:0/2429758403 conn(0x562377649fe0 msgr2=0x56237767fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:19.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.698+0000 7fba0ebc6740 1 -- 172.21.15.102:0/2429758403 shutdown_connections 2024-09-13T17:42:19.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.698+0000 7fba0ebc6740 1 -- 172.21.15.102:0/2429758403 wait complete. 2024-09-13T17:42:19.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.698+0000 7fba0ebc6740 1 -- start start 2024-09-13T17:42:19.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.698+0000 7fba0ebc6740 1 Processor -- start 2024-09-13T17:42:19.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.699+0000 7fba0ebc6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562377699960 0x56237769ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:19.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.699+0000 7fba0ebc6740 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562377699d30 0x56237769f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:19.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.699+0000 7fba0ebc6740 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5623776a6350 0x56237769fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:19.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.699+0000 7fba0ebc6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623775841b0 con 0x562377699960 2024-09-13T17:42:19.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.699+0000 7fba0ebc6740 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x562377583e50 con 0x562377699d30 2024-09-13T17:42:19.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.699+0000 7fba0ebc6740 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x56237732c590 con 0x5623776a6350 2024-09-13T17:42:19.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.699+0000 7fba0ebc6740 10 client.0 _ll_register_callbacks cb 0x5623775dbb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:42:19.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.699+0000 7fba0a3eb640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562377699960 0x56237769ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:19.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.699+0000 7fba09bea640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562377699d30 0x56237769f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:19.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.699+0000 7fba0a3eb640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562377699960 0x56237769ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37714/0 (socket says 172.21.15.102:37714) 2024-09-13T17:42:19.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.699+0000 7fba0a3eb640 1 -- 172.21.15.102:0/3004010017 learned_addr learned my addr 172.21.15.102:0/3004010017 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:19.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.699+0000 7fba0abec640 1 --2- 172.21.15.102:0/3004010017 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5623776a6350 0x56237769fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:19.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.699+0000 7fba09bea640 1 -- 172.21.15.102:0/3004010017 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5623776a6350 msgr2=0x56237769fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:19.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.699+0000 7fba09bea640 1 --2- 172.21.15.102:0/3004010017 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5623776a6350 0x56237769fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.699+0000 7fba09bea640 1 -- 172.21.15.102:0/3004010017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562377699960 msgr2=0x56237769ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:19.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.699+0000 7fba09bea640 1 --2- 172.21.15.102:0/3004010017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562377699960 0x56237769ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.699+0000 7fba09bea640 1 -- 172.21.15.102:0/3004010017 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba04120520 con 0x562377699d30 2024-09-13T17:42:19.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.699+0000 7fba0a3eb640 1 --2- 172.21.15.102:0/3004010017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562377699960 0x56237769ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:19.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.699+0000 7fba09bea640 1 --2- 172.21.15.102:0/3004010017 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562377699d30 0x56237769f4c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x56237769df90 tx=0x7fba00007e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e212d73f7d9b70e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:19.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.699+0000 7fb9e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.102:3301/0 2024-09-13T17:42:19.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.699+0000 7fb9e4ff9640 1 -- 172.21.15.102:0/3004010017 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba00007680 con 0x562377699d30 2024-09-13T17:42:19.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.699+0000 7fb9e4ff9640 1 -- 172.21.15.102:0/3004010017 <== mon.1 v2:172.21.15.102:3301/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7fba000060d0 con 0x562377699d30 2024-09-13T17:42:19.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.699+0000 7fba0ebc6740 1 -- 172.21.15.102:0/3004010017 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5623772e83c0 con 0x562377699d30 2024-09-13T17:42:19.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.700+0000 7fb9e4ff9640 1 -- 172.21.15.102:0/3004010017 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_get_version_reply(handle=1 version=5) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba000064e0 con 0x562377699d30 2024-09-13T17:42:19.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.700+0000 7fba0ebc6740 10 client.4507 fetch_fsmap learned FSMap version 5 2024-09-13T17:42:19.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.700+0000 7fba0ebc6740 1 -- 172.21.15.102:0/3004010017 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,fsmap=5,monmap=2+}) v3 -- 0x562377594ab0 con 0x562377699d30 2024-09-13T17:42:19.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.700+0000 7fb9e4ff9640 1 -- 172.21.15.102:0/3004010017 <== mon.1 v2:172.21.15.102:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5623772e83c0 con 0x562377699d30 2024-09-13T17:42:19.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.700+0000 7fb9e4ff9640 1 -- 172.21.15.102:0/3004010017 <== mon.1 v2:172.21.15.102:3301/0 5 ==== fsmap(e 5) v1 ==== 1430+0+0 (secure 0 0 0) 0x7fba000147b0 con 0x562377699d30 2024-09-13T17:42:19.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.700+0000 7fba0ebc6740 10 client.4507 fetch_fsmap finished waiting for FSMap version 5 2024-09-13T17:42:19.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.700+0000 7fba0ebc6740 10 client.4507 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-13T17:42:19.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.700+0000 7fba0ebc6740 20 client.4507 populate_metadata read hostname 'smithi102' 2024-09-13T17:42:19.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.700+0000 7fba0ebc6740 4 client.4507 mds_command: new command op to 4210 tid=0[{"prefix": "get_command_descriptions"}] 2024-09-13T17:42:19.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.700+0000 7fba0ebc6740 1 -- 172.21.15.102:0/3004010017 --> v1:172.21.15.102:6817/3443280887 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5623772f96c0 con 0x5623776b4f20 2024-09-13T17:42:19.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.700+0000 7fb9e4ff9640 10 client.4507 ms_handle_connect on v1:172.21.15.102:6817/3443280887 2024-09-13T17:42:19.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.701+0000 7fb9e4ff9640 1 -- 172.21.15.102:0/3004010017 <== mds.0 v1:172.21.15.102:6817/3443280887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17536 (unknown 0 0 610023333) 0x5623772f96c0 con 0x5623776b4f20 2024-09-13T17:42:19.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.701+0000 7fb9e4ff9640 10 client.4507 handle_command_reply: tid=0 2024-09-13T17:42:19.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.701+0000 7fba0ebc6740 1 client.4507 shutdown 2024-09-13T17:42:19.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.701+0000 7fba0ebc6740 1 -- 172.21.15.102:0/3004010017 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562377699d30 msgr2=0x56237769f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:19.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.701+0000 7fba0ebc6740 1 --2- 172.21.15.102:0/3004010017 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562377699d30 0x56237769f4c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x56237769df90 tx=0x7fba00007e40 comp rx=0 tx=0).stop 2024-09-13T17:42:19.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.701+0000 7fba0ebc6740 1 -- 172.21.15.102:0/3004010017 shutdown_connections 2024-09-13T17:42:19.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.701+0000 7fba0ebc6740 1 --2- 172.21.15.102:0/3004010017 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5623776a6350 0x56237769fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.701+0000 7fba0ebc6740 1 --2- 172.21.15.102:0/3004010017 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562377699d30 0x56237769f4c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.701+0000 7fba0ebc6740 1 --2- 172.21.15.102:0/3004010017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562377699960 0x56237769ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.701+0000 7fb9e4ff9640 0 client.4507 ms_handle_reset on v1:172.21.15.102:6817/3443280887 2024-09-13T17:42:19.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.701+0000 7fba0ebc6740 1 -- 172.21.15.102:0/3004010017 >> 172.21.15.102:0/3004010017 conn(0x562377649fe0 msgr2=0x56237767bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:19.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.701+0000 7fba0ebc6740 1 -- 172.21.15.102:0/3004010017 shutdown_connections 2024-09-13T17:42:19.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.701+0000 7fba0ebc6740 1 -- 172.21.15.102:0/3004010017 wait complete. 2024-09-13T17:42:19.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.701+0000 7fba0ebc6740 20 client.4507 trim_cache size 0 max 16384 2024-09-13T17:42:19.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.715+0000 7fba0ebc6740 1 Processor -- start 2024-09-13T17:42:19.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.715+0000 7fba0ebc6740 1 -- start start 2024-09-13T17:42:19.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.716+0000 7fba0ebc6740 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x562377699960 0x5623776b8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:19.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.716+0000 7fba0ebc6740 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562377699d30 0x5623776b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:19.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.716+0000 7fba0ebc6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5623776a6350 0x5623776b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:19.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.716+0000 7fba0ebc6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562377680b50 con 0x5623776a6350 2024-09-13T17:42:19.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.716+0000 7fba0ebc6740 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x5623772f96c0 con 0x562377699d30 2024-09-13T17:42:19.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.716+0000 7fba0ebc6740 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x56237767e0c0 con 0x562377699960 2024-09-13T17:42:19.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.716+0000 7fba0a3eb640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x562377699960 0x5623776b8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:19.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.716+0000 7fba09bea640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562377699d30 0x5623776b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:19.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.716+0000 7fba0abec640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5623776a6350 0x5623776b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:19.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.716+0000 7fba0a3eb640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x562377699960 0x5623776b8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:46988/0 (socket says 172.21.15.102:46988) 2024-09-13T17:42:19.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.716+0000 7fba09bea640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562377699d30 0x5623776b1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35756/0 (socket says 172.21.15.102:35756) 2024-09-13T17:42:19.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.716+0000 7fba0abec640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5623776a6350 0x5623776b2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37728/0 (socket says 172.21.15.102:37728) 2024-09-13T17:42:19.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.716+0000 7fba0a3eb640 1 -- 172.21.15.102:0/1514251670 learned_addr learned my addr 172.21.15.102:0/1514251670 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:19.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.716+0000 7fba0a3eb640 1 -- 172.21.15.102:0/1514251670 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562377699d30 msgr2=0x5623776b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:19.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.716+0000 7fba0a3eb640 1 --2- 172.21.15.102:0/1514251670 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562377699d30 0x5623776b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.716+0000 7fba0a3eb640 1 -- 172.21.15.102:0/1514251670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5623776a6350 msgr2=0x5623776b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:19.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.716+0000 7fba0a3eb640 1 --2- 172.21.15.102:0/1514251670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5623776a6350 0x5623776b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.716+0000 7fba0a3eb640 1 -- 172.21.15.102:0/1514251670 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562377588bc0 con 0x562377699960 2024-09-13T17:42:19.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.716+0000 7fba0a3eb640 1 --2- 172.21.15.102:0/1514251670 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x562377699960 0x5623776b8310 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb9f801e000 tx=0x7fb9f8033040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c9a7d6fd2dafe8e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:19.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.716+0000 7fb9e7fff640 1 -- 172.21.15.102:0/1514251670 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8023870 con 0x562377699960 2024-09-13T17:42:19.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.716+0000 7fb9e7fff640 1 -- 172.21.15.102:0/1514251670 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f80135d0 con 0x562377699960 2024-09-13T17:42:19.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.716+0000 7fb9e7fff640 1 -- 172.21.15.102:0/1514251670 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f80125e0 con 0x562377699960 2024-09-13T17:42:19.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.717+0000 7fba0ebc6740 1 -- 172.21.15.102:0/1514251670 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x562377699960 msgr2=0x5623776b8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:19.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.717+0000 7fba0ebc6740 1 --2- 172.21.15.102:0/1514251670 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x562377699960 0x5623776b8310 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb9f801e000 tx=0x7fb9f8033040 comp rx=0 tx=0).stop 2024-09-13T17:42:19.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.717+0000 7fba0ebc6740 1 -- 172.21.15.102:0/1514251670 shutdown_connections 2024-09-13T17:42:19.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.717+0000 7fba0ebc6740 1 --2- 172.21.15.102:0/1514251670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5623776a6350 0x5623776b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.717+0000 7fba0ebc6740 1 --2- 172.21.15.102:0/1514251670 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562377699d30 0x5623776b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.717+0000 7fba0ebc6740 1 --2- 172.21.15.102:0/1514251670 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x562377699960 0x5623776b8310 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.717+0000 7fba0ebc6740 1 -- 172.21.15.102:0/1514251670 >> 172.21.15.102:0/1514251670 conn(0x562377649fe0 msgr2=0x56237767fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:19.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.717+0000 7fba0ebc6740 1 -- 172.21.15.102:0/1514251670 shutdown_connections 2024-09-13T17:42:19.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.717+0000 7fba0ebc6740 1 -- 172.21.15.102:0/1514251670 wait complete. 2024-09-13T17:42:19.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.717+0000 7fba0ebc6740 1 -- start start 2024-09-13T17:42:19.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.717+0000 7fba0ebc6740 1 Processor -- start 2024-09-13T17:42:19.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.718+0000 7fba0ebc6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562377699960 0x5623776aa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:19.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.718+0000 7fba0ebc6740 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562377699d30 0x5623776b7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:19.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.718+0000 7fba0ebc6740 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5623776a6350 0x5623776b7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:19.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.718+0000 7fba0ebc6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623776a9650 con 0x562377699960 2024-09-13T17:42:19.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.718+0000 7fba0ebc6740 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x5623776b1650 con 0x562377699d30 2024-09-13T17:42:19.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.718+0000 7fba0ebc6740 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x5623776b8700 con 0x5623776a6350 2024-09-13T17:42:19.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.718+0000 7fba0ebc6740 10 client.0 _ll_register_callbacks cb 0x7fba000075d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:42:19.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.718+0000 7fba0a3eb640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562377699960 0x5623776aa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:19.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.718+0000 7fba0abec640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5623776a6350 0x5623776b7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:19.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.718+0000 7fba09bea640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562377699d30 0x5623776b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:19.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.718+0000 7fba0abec640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5623776a6350 0x5623776b7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:46994/0 (socket says 172.21.15.102:46994) 2024-09-13T17:42:19.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.718+0000 7fba0a3eb640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562377699960 0x5623776aa1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37732/0 (socket says 172.21.15.102:37732) 2024-09-13T17:42:19.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.718+0000 7fba0abec640 1 -- 172.21.15.102:0/3041357868 learned_addr learned my addr 172.21.15.102:0/3041357868 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:19.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.718+0000 7fba0abec640 1 -- 172.21.15.102:0/3041357868 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562377699d30 msgr2=0x5623776b7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:19.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.718+0000 7fba0abec640 1 --2- 172.21.15.102:0/3041357868 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562377699d30 0x5623776b7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.718+0000 7fba0abec640 1 -- 172.21.15.102:0/3041357868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562377699960 msgr2=0x5623776aa1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:19.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.718+0000 7fba0abec640 1 --2- 172.21.15.102:0/3041357868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562377699960 0x5623776aa1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.718+0000 7fba0abec640 1 -- 172.21.15.102:0/3041357868 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56237730a9e0 con 0x5623776a6350 2024-09-13T17:42:19.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.718+0000 7fba0abec640 1 --2- 172.21.15.102:0/3041357868 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5623776a6350 0x5623776b7840 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc009880 tx=0x7fb9fc036000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e2a6ae46cf73abd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:19.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.718+0000 7fb9e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.102:3302/0 2024-09-13T17:42:19.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.718+0000 7fb9e4ff9640 1 -- 172.21.15.102:0/3041357868 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc041040 con 0x5623776a6350 2024-09-13T17:42:19.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.719+0000 7fb9e4ff9640 1 -- 172.21.15.102:0/3041357868 <== mon.2 v2:172.21.15.102:3302/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7fb9fc048440 con 0x5623776a6350 2024-09-13T17:42:19.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.719+0000 7fba0ebc6740 1 -- 172.21.15.102:0/3041357868 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9f8023e60 con 0x5623776a6350 2024-09-13T17:42:19.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.719+0000 7fb9e4ff9640 1 -- 172.21.15.102:0/3041357868 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_get_version_reply(handle=1 version=5) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9fc048810 con 0x5623776a6350 2024-09-13T17:42:19.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.719+0000 7fba0ebc6740 10 client.4505 fetch_fsmap learned FSMap version 5 2024-09-13T17:42:19.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.719+0000 7fba0ebc6740 1 -- 172.21.15.102:0/3041357868 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,fsmap=5,monmap=2+}) v3 -- 0x5623775ae780 con 0x5623776a6350 2024-09-13T17:42:19.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.719+0000 7fb9e4ff9640 1 -- 172.21.15.102:0/3041357868 <== mon.2 v2:172.21.15.102:3302/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f8023e60 con 0x5623776a6350 2024-09-13T17:42:19.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.719+0000 7fb9e4ff9640 1 -- 172.21.15.102:0/3041357868 <== mon.2 v2:172.21.15.102:3302/0 5 ==== fsmap(e 5) v1 ==== 1430+0+0 (secure 0 0 0) 0x7fb9fc052450 con 0x5623776a6350 2024-09-13T17:42:19.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.719+0000 7fba0ebc6740 10 client.4505 fetch_fsmap finished waiting for FSMap version 5 2024-09-13T17:42:19.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.719+0000 7fba0ebc6740 10 client.4505 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-13T17:42:19.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.719+0000 7fba0ebc6740 20 client.4505 populate_metadata read hostname 'smithi102' 2024-09-13T17:42:19.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.719+0000 7fba0ebc6740 4 client.4505 mds_command: new command op to 4210 tid=0[{"prefix": "status"}] 2024-09-13T17:42:19.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.719+0000 7fba0ebc6740 1 -- 172.21.15.102:0/3041357868 --> v1:172.21.15.102:6817/3443280887 -- command(tid 0: {"prefix": "status"}) v1 -- 0x56237734cd10 con 0x56237770ae00 2024-09-13T17:42:19.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.719+0000 7fb9e4ff9640 10 client.4505 ms_handle_connect on v1:172.21.15.102:6817/3443280887 2024-09-13T17:42:19.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.719+0000 7fb9e4ff9640 1 -- 172.21.15.102:0/3041357868 <== mds.0 v1:172.21.15.102:6817/3443280887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (unknown 0 0 2087724827) 0x56237734cd10 con 0x56237770ae00 2024-09-13T17:42:19.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.719+0000 7fb9e4ff9640 10 client.4505 handle_command_reply: tid=0 2024-09-13T17:42:19.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.719+0000 7fba0ebc6740 1 client.4505 shutdown 2024-09-13T17:42:19.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.720+0000 7fba0ebc6740 1 -- 172.21.15.102:0/3041357868 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5623776a6350 msgr2=0x5623776b7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:19.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.720+0000 7fba0ebc6740 1 --2- 172.21.15.102:0/3041357868 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5623776a6350 0x5623776b7840 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc009880 tx=0x7fb9fc036000 comp rx=0 tx=0).stop 2024-09-13T17:42:19.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.720+0000 7fba0ebc6740 1 -- 172.21.15.102:0/3041357868 shutdown_connections 2024-09-13T17:42:19.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.720+0000 7fba0ebc6740 1 --2- 172.21.15.102:0/3041357868 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5623776a6350 0x5623776b7840 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.720+0000 7fba0ebc6740 1 --2- 172.21.15.102:0/3041357868 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x562377699d30 0x5623776b7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.720+0000 7fb9e4ff9640 0 client.4505 ms_handle_reset on v1:172.21.15.102:6817/3443280887 2024-09-13T17:42:19.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.720+0000 7fba0ebc6740 1 --2- 172.21.15.102:0/3041357868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562377699960 0x5623776aa1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.720+0000 7fba0ebc6740 1 -- 172.21.15.102:0/3041357868 >> 172.21.15.102:0/3041357868 conn(0x562377649fe0 msgr2=0x56237769a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:19.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.720+0000 7fba0ebc6740 1 -- 172.21.15.102:0/3041357868 shutdown_connections 2024-09-13T17:42:19.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.720+0000 7fba0ebc6740 1 -- 172.21.15.102:0/3041357868 wait complete. 2024-09-13T17:42:19.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.720+0000 7fba0ebc6740 20 client.4505 trim_cache size 0 max 16384 2024-09-13T17:42:19.728 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-13T17:42:19.728 INFO:teuthology.orchestra.run.smithi102.stdout: "cluster_fsid": "6507a4f0-6c1b-42c0-9c4d-5af24a57491d", 2024-09-13T17:42:19.728 INFO:teuthology.orchestra.run.smithi102.stdout: "whoami": 0, 2024-09-13T17:42:19.728 INFO:teuthology.orchestra.run.smithi102.stdout: "id": 4210, 2024-09-13T17:42:19.729 INFO:teuthology.orchestra.run.smithi102.stdout: "want_state": "up:active", 2024-09-13T17:42:19.729 INFO:teuthology.orchestra.run.smithi102.stdout: "state": "up:active", 2024-09-13T17:42:19.729 INFO:teuthology.orchestra.run.smithi102.stdout: "fs_name": "cephfs", 2024-09-13T17:42:19.729 INFO:teuthology.orchestra.run.smithi102.stdout: "rank_uptime": 12.418434134, 2024-09-13T17:42:19.729 INFO:teuthology.orchestra.run.smithi102.stdout: "mdsmap_epoch": 5, 2024-09-13T17:42:19.729 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_epoch": 20, 2024-09-13T17:42:19.729 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_epoch_barrier": 20, 2024-09-13T17:42:19.729 INFO:teuthology.orchestra.run.smithi102.stdout: "uptime": 17.450417934000001 2024-09-13T17:42:19.729 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-13T17:42:19.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.720+0000 7fba0c64f640 1 -- 172.21.15.102:0/1611958714 >> v1:172.21.15.102:6816/34937 conn(0x7fb9e8031160 legacy=0x7fb9e8033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:19.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.720+0000 7fba0c64f640 1 -- 172.21.15.102:0/1611958714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba04067130 msgr2=0x7fba0413e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:19.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.720+0000 7fba0c64f640 1 --2- 172.21.15.102:0/1611958714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba04067130 0x7fba0413e760 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800ec80 tx=0x7fb9f80090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.720+0000 7fba0c64f640 1 -- 172.21.15.102:0/1611958714 shutdown_connections 2024-09-13T17:42:19.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.720+0000 7fba0c64f640 1 --2- 172.21.15.102:0/1611958714 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fba04098ef0 0x7fba0413f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.720+0000 7fba0c64f640 1 --2- 172.21.15.102:0/1611958714 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fba04067c30 0x7fba0413eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.720+0000 7fba0c64f640 1 --2- 172.21.15.102:0/1611958714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba04067130 0x7fba0413e760 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.720+0000 7fba0c64f640 1 -- 172.21.15.102:0/1611958714 >> 172.21.15.102:0/1611958714 conn(0x7fba0405db50 msgr2=0x7fba0406cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:19.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.720+0000 7fba0c64f640 1 -- 172.21.15.102:0/1611958714 shutdown_connections 2024-09-13T17:42:19.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.720+0000 7fba0c64f640 1 -- 172.21.15.102:0/1611958714 wait complete. 2024-09-13T17:42:19.733 INFO:teuthology.run_tasks:Running task exec... 2024-09-13T17:42:19.746 INFO:teuthology.task.exec:Executing custom commands... 2024-09-13T17:42:19.746 INFO:teuthology.task.exec:Running commands on role osd.0 host ubuntu@smithi102.front.sepia.ceph.com 2024-09-13T17:42:19.747 DEBUG:teuthology.orchestra.run.smithi102:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd set-require-min-compat-client quincy' 2024-09-13T17:42:19.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.837+0000 7fab4619d640 1 Processor -- start 2024-09-13T17:42:19.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.837+0000 7fab4619d640 1 -- start start 2024-09-13T17:42:19.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.837+0000 7fab4619d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fab40096dd0 0x7fab4009d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:19.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.837+0000 7fab4619d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fab40130f70 0x7fab40133350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:19.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.837+0000 7fab4619d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab400982b0 con 0x7fab40098a40 2024-09-13T17:42:19.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.837+0000 7fab4619d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab40097280 con 0x7fab40096dd0 2024-09-13T17:42:19.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.837+0000 7fab4619d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab4006ea50 con 0x7fab40130f70 2024-09-13T17:42:19.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.837+0000 7fab3ffff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fab40130f70 0x7fab40133350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:19.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.837+0000 7fab3effd640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fab40096dd0 0x7fab4009d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:19.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.838+0000 7fab3f7fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fab40098a40 0x7fab4009cf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:60978/0 (socket says 172.21.15.102:60978) 2024-09-13T17:42:19.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.838+0000 7fab3f7fe640 1 -- 172.21.15.102:0/3231777450 learned_addr learned my addr 172.21.15.102:0/3231777450 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:42:19.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.838+0000 7fab3effd640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fab40096dd0 0x7fab4009d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37736/0 (socket says 172.21.15.102:37736) 2024-09-13T17:42:19.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.838+0000 7fab3ffff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fab40130f70 0x7fab40133350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35774/0 (socket says 172.21.15.102:35774) 2024-09-13T17:42:19.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.838+0000 7fab3ffff640 1 -- 172.21.15.102:0/3231777450 >> v2:172.21.15.102:3300/0 conn(0x7fab40096dd0 msgr2=0x7fab4009d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:19.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.838+0000 7fab3ffff640 1 --2- 172.21.15.102:0/3231777450 >> v2:172.21.15.102:3300/0 conn(0x7fab40096dd0 0x7fab4009d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.838+0000 7fab3ffff640 1 -- 172.21.15.102:0/3231777450 >> v1:172.21.15.102:6789/0 conn(0x7fab40098a40 legacy=0x7fab4009cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:19.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.838+0000 7fab3ffff640 1 -- 172.21.15.102:0/3231777450 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab40097f80 con 0x7fab40130f70 2024-09-13T17:42:19.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.838+0000 7fab3effd640 1 --2- 172.21.15.102:0/3231777450 >> v2:172.21.15.102:3300/0 conn(0x7fab40096dd0 0x7fab4009d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:19.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.839+0000 7fab3ffff640 1 --2- 172.21.15.102:0/3231777450 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fab40130f70 0x7fab40133350 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fab38009e90 tx=0x7fab38009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e38a5c8a935d5b12 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:19.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.839+0000 7fab3e7fc640 1 -- 172.21.15.102:0/3231777450 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab38007550 con 0x7fab40130f70 2024-09-13T17:42:19.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.839+0000 7fab3e7fc640 1 -- 172.21.15.102:0/3231777450 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3802d050 con 0x7fab40130f70 2024-09-13T17:42:19.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.839+0000 7fab3e7fc640 1 -- 172.21.15.102:0/3231777450 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab38007920 con 0x7fab40130f70 2024-09-13T17:42:19.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.839+0000 7fab4619d640 1 -- 172.21.15.102:0/3231777450 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fab40130f70 msgr2=0x7fab40133350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:19.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.839+0000 7fab4619d640 1 --2- 172.21.15.102:0/3231777450 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fab40130f70 0x7fab40133350 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fab38009e90 tx=0x7fab38009780 comp rx=0 tx=0).stop 2024-09-13T17:42:19.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.839+0000 7fab4619d640 1 -- 172.21.15.102:0/3231777450 shutdown_connections 2024-09-13T17:42:19.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.839+0000 7fab4619d640 1 --2- 172.21.15.102:0/3231777450 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fab40130f70 0x7fab40133350 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.839+0000 7fab4619d640 1 --2- 172.21.15.102:0/3231777450 >> v2:172.21.15.102:3300/0 conn(0x7fab40096dd0 0x7fab4009d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.839+0000 7fab4619d640 1 -- 172.21.15.102:0/3231777450 >> 172.21.15.102:0/3231777450 conn(0x7fab4009e770 msgr2=0x7fab4009eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:19.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.839+0000 7fab4619d640 1 -- 172.21.15.102:0/3231777450 shutdown_connections 2024-09-13T17:42:19.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.839+0000 7fab4619d640 1 -- 172.21.15.102:0/3231777450 wait complete. 2024-09-13T17:42:19.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.840+0000 7fab4619d640 1 Processor -- start 2024-09-13T17:42:19.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.840+0000 7fab4619d640 1 -- start start 2024-09-13T17:42:19.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.841+0000 7fab4619d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fab40096dd0 0x7fab4012f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:19.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.841+0000 7fab4619d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fab40098a40 0x7fab4011b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:19.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.841+0000 7fab4619d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab40130f70 0x7fab4011bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:19.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.841+0000 7fab4619d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4011f130 con 0x7fab40130f70 2024-09-13T17:42:19.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.841+0000 7fab4619d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab4011d300 con 0x7fab40098a40 2024-09-13T17:42:19.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.841+0000 7fab4619d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fab4006b590 con 0x7fab40096dd0 2024-09-13T17:42:19.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.841+0000 7fab3f7fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fab40096dd0 0x7fab4012f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:19.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.841+0000 7fab3effd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fab40098a40 0x7fab4011b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:19.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.841+0000 7fab3ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab40130f70 0x7fab4011bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:19.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.841+0000 7fab3effd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fab40098a40 0x7fab4011b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35782/0 (socket says 172.21.15.102:35782) 2024-09-13T17:42:19.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.841+0000 7fab3f7fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fab40096dd0 0x7fab4012f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47004/0 (socket says 172.21.15.102:47004) 2024-09-13T17:42:19.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.841+0000 7fab3effd640 1 -- 172.21.15.102:0/2658744071 learned_addr learned my addr 172.21.15.102:0/2658744071 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:19.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.841+0000 7fab3effd640 1 -- 172.21.15.102:0/2658744071 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fab40096dd0 msgr2=0x7fab4012f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:19.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.841+0000 7fab3effd640 1 --2- 172.21.15.102:0/2658744071 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fab40096dd0 0x7fab4012f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.841+0000 7fab3effd640 1 -- 172.21.15.102:0/2658744071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab40130f70 msgr2=0x7fab4011bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:19.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.841+0000 7fab3effd640 1 --2- 172.21.15.102:0/2658744071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab40130f70 0x7fab4011bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:19.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.841+0000 7fab3effd640 1 -- 172.21.15.102:0/2658744071 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab3400a530 con 0x7fab40098a40 2024-09-13T17:42:19.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.841+0000 7fab3ffff640 1 --2- 172.21.15.102:0/2658744071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab40130f70 0x7fab4011bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:19.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.841+0000 7fab3f7fe640 1 --2- 172.21.15.102:0/2658744071 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fab40096dd0 0x7fab4012f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:19.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.841+0000 7fab3effd640 1 --2- 172.21.15.102:0/2658744071 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fab40098a40 0x7fab4011b850 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fab3400e970 tx=0x7fab3400a2b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:19.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.841+0000 7fab277fe640 1 -- 172.21.15.102:0/2658744071 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3400ca40 con 0x7fab40098a40 2024-09-13T17:42:19.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.841+0000 7fab277fe640 1 -- 172.21.15.102:0/2658744071 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3400cba0 con 0x7fab40098a40 2024-09-13T17:42:19.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.841+0000 7fab4619d640 1 -- 172.21.15.102:0/2658744071 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab38009a20 con 0x7fab40098a40 2024-09-13T17:42:19.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.841+0000 7fab277fe640 1 -- 172.21.15.102:0/2658744071 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3401a430 con 0x7fab40098a40 2024-09-13T17:42:19.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.841+0000 7fab4619d640 1 -- 172.21.15.102:0/2658744071 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab4011da10 con 0x7fab40098a40 2024-09-13T17:42:19.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.842+0000 7fab277fe640 1 -- 172.21.15.102:0/2658744071 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fab340113f0 con 0x7fab40098a40 2024-09-13T17:42:19.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.842+0000 7fab4619d640 1 -- 172.21.15.102:0/2658744071 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab40067e90 con 0x7fab40098a40 2024-09-13T17:42:19.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.843+0000 7fab277fe640 1 -- 172.21.15.102:0/2658744071 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3688+0+0 (secure 0 0 0) 0x7fab34015030 con 0x7fab40098a40 2024-09-13T17:42:19.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:19.847+0000 7fab277fe640 1 -- 172.21.15.102:0/2658744071 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fab340193e0 con 0x7fab40098a40 2024-09-13T17:42:20.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.005+0000 7fab4619d640 1 -- 172.21.15.102:0/2658744071 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd set-require-min-compat-client", "version": "quincy"} v 0) v1 -- 0x7fab400717d0 con 0x7fab40098a40 2024-09-13T17:42:20.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.340+0000 7fab277fe640 1 -- 172.21.15.102:0/2658744071 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd set-require-min-compat-client", "version": "quincy"}]=0 set require_min_compat_client to quincy v21) v1 ==== 141+0+0 (secure 0 0 0) 0x7fab340212f0 con 0x7fab40098a40 2024-09-13T17:42:20.342 INFO:teuthology.orchestra.run.smithi102.stderr:set require_min_compat_client to quincy 2024-09-13T17:42:20.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.342+0000 7fab4619d640 1 -- 172.21.15.102:0/2658744071 >> v1:172.21.15.102:6816/34937 conn(0x7fab2c031160 legacy=0x7fab2c033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:20.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.342+0000 7fab4619d640 1 -- 172.21.15.102:0/2658744071 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fab40098a40 msgr2=0x7fab4011b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:20.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.342+0000 7fab4619d640 1 --2- 172.21.15.102:0/2658744071 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fab40098a40 0x7fab4011b850 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fab3400e970 tx=0x7fab3400a2b0 comp rx=0 tx=0).stop 2024-09-13T17:42:20.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.342+0000 7fab4619d640 1 -- 172.21.15.102:0/2658744071 shutdown_connections 2024-09-13T17:42:20.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.342+0000 7fab4619d640 1 --2- 172.21.15.102:0/2658744071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab40130f70 0x7fab4011bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:20.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.342+0000 7fab4619d640 1 --2- 172.21.15.102:0/2658744071 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fab40098a40 0x7fab4011b850 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:20.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.342+0000 7fab4619d640 1 --2- 172.21.15.102:0/2658744071 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fab40096dd0 0x7fab4012f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:20.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.342+0000 7fab4619d640 1 -- 172.21.15.102:0/2658744071 >> 172.21.15.102:0/2658744071 conn(0x7fab4009e770 msgr2=0x7fab401302a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:20.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.343+0000 7fab4619d640 1 -- 172.21.15.102:0/2658744071 shutdown_connections 2024-09-13T17:42:20.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.343+0000 7fab4619d640 1 -- 172.21.15.102:0/2658744071 wait complete. 2024-09-13T17:42:20.356 INFO:teuthology.run_tasks:Running task print... 2024-09-13T17:42:20.365 INFO:teuthology.task.print:**** done ceph 2024-09-13T17:42:20.365 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-13T17:42:20.424 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-13T17:42:20.425 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-09-13T17:42:20.425 INFO:tasks.ceph_fuse:client.0 config is {} 2024-09-13T17:42:20.425 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-13T17:42:20.425 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T17:42:20.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:20.425 DEBUG:teuthology.orchestra.run.smithi171:> ip netns list 2024-09-13T17:42:20.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:20.446 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip link delete ceph-brx 2024-09-13T17:42:20.519 INFO:teuthology.orchestra.run.smithi171.stderr:Cannot find device "ceph-brx" 2024-09-13T17:42:20.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:42:20.520 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-13T17:42:20.521 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-13T17:42:20.521 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T17:42:20.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.614+0000 7f77a595d640 1 Processor -- start 2024-09-13T17:42:20.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.614+0000 7f77a595d640 1 -- start start 2024-09-13T17:42:20.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.615+0000 7f77a595d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77a0098ef0 0x7f77a00992c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:20.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.615+0000 7f77a595d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f77a0067130 0x7f77a0067520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:20.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.615+0000 7f77a595d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77a009a0e0 con 0x7f77a0067a60 2024-09-13T17:42:20.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.615+0000 7f77a595d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a0097600 con 0x7f77a0067130 2024-09-13T17:42:20.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.615+0000 7f77a595d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f77a0096ea0 con 0x7f77a0098ef0 2024-09-13T17:42:20.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.615+0000 7f779effd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77a0098ef0 0x7f77a00992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:20.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.615+0000 7f779e7fc640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f77a0067130 0x7f77a0067520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:20.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.615+0000 7f779e7fc640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f77a0067130 0x7f77a0067520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35790/0 (socket says 172.21.15.102:35790) 2024-09-13T17:42:20.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.615+0000 7f779f7fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f77a0067a60 0x7f77a0136750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:60992/0 (socket says 172.21.15.102:60992) 2024-09-13T17:42:20.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.615+0000 7f779effd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77a0098ef0 0x7f77a00992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47012/0 (socket says 172.21.15.102:47012) 2024-09-13T17:42:20.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.615+0000 7f779e7fc640 1 -- 172.21.15.102:0/2760372474 learned_addr learned my addr 172.21.15.102:0/2760372474 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:20.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.615+0000 7f779dffb640 1 -- 172.21.15.102:0/2760372474 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1196378520 0 0) 0x7f77a009a0e0 con 0x7f77a0067a60 2024-09-13T17:42:20.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.615+0000 7f779dffb640 1 -- 172.21.15.102:0/2760372474 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77880034e0 con 0x7f77a0067a60 2024-09-13T17:42:20.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.615+0000 7f779effd640 1 -- 172.21.15.102:0/2760372474 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f77a0067130 msgr2=0x7f77a0067520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:20.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.615+0000 7f779effd640 1 --2- 172.21.15.102:0/2760372474 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f77a0067130 0x7f77a0067520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:20.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.615+0000 7f779effd640 1 -- 172.21.15.102:0/2760372474 >> v1:172.21.15.102:6789/0 conn(0x7f77a0067a60 legacy=0x7f77a0136750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:20.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.615+0000 7f779effd640 1 -- 172.21.15.102:0/2760372474 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a0097cc0 con 0x7f77a0098ef0 2024-09-13T17:42:20.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.615+0000 7f779e7fc640 1 --2- 172.21.15.102:0/2760372474 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f77a0067130 0x7f77a0067520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:20.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.616+0000 7f779effd640 1 --2- 172.21.15.102:0/2760372474 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77a0098ef0 0x7f77a00992c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7790009880 tx=0x7f77900090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aac84d5a5d7e6808 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:20.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.616+0000 7f779dffb640 1 -- 172.21.15.102:0/2760372474 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779002ac50 con 0x7f77a0098ef0 2024-09-13T17:42:20.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.616+0000 7f779dffb640 1 -- 172.21.15.102:0/2760372474 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f779002adb0 con 0x7f77a0098ef0 2024-09-13T17:42:20.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.616+0000 7f779dffb640 1 -- 172.21.15.102:0/2760372474 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7790032670 con 0x7f77a0098ef0 2024-09-13T17:42:20.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.616+0000 7f77a595d640 1 -- 172.21.15.102:0/2760372474 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77a0098ef0 msgr2=0x7f77a00992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:20.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.616+0000 7f77a595d640 1 --2- 172.21.15.102:0/2760372474 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77a0098ef0 0x7f77a00992c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7790009880 tx=0x7f77900090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:20.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.616+0000 7f77a595d640 1 -- 172.21.15.102:0/2760372474 shutdown_connections 2024-09-13T17:42:20.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.616+0000 7f77a595d640 1 --2- 172.21.15.102:0/2760372474 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f77a0067130 0x7f77a0067520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:20.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.616+0000 7f77a595d640 1 --2- 172.21.15.102:0/2760372474 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77a0098ef0 0x7f77a00992c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:20.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.616+0000 7f77a595d640 1 -- 172.21.15.102:0/2760372474 >> 172.21.15.102:0/2760372474 conn(0x7f77a005db50 msgr2=0x7f77a005df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:20.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.617+0000 7f77a595d640 1 -- 172.21.15.102:0/2760372474 shutdown_connections 2024-09-13T17:42:20.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.617+0000 7f77a595d640 1 -- 172.21.15.102:0/2760372474 wait complete. 2024-09-13T17:42:20.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.617+0000 7f77a595d640 1 Processor -- start 2024-09-13T17:42:20.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.617+0000 7f77a595d640 1 -- start start 2024-09-13T17:42:20.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.618+0000 7f77a595d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77a0067130 0x7f77a011f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:20.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.618+0000 7f77a595d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f77a0067a60 0x7f77a011c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:20.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.618+0000 7f77a595d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77a0098ef0 0x7f77a011c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:20.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.618+0000 7f77a595d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a009dda0 con 0x7f77a0098ef0 2024-09-13T17:42:20.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.618+0000 7f77a595d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a009b460 con 0x7f77a0067a60 2024-09-13T17:42:20.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.618+0000 7f77a595d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f77a011edb0 con 0x7f77a0067130 2024-09-13T17:42:20.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.618+0000 7f779effd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77a0067130 0x7f77a011f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:20.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.618+0000 7f779e7fc640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f77a0067a60 0x7f77a011c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:20.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.618+0000 7f779f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77a0098ef0 0x7f77a011c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:20.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.618+0000 7f779effd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77a0067130 0x7f77a011f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47014/0 (socket says 172.21.15.102:47014) 2024-09-13T17:42:20.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.618+0000 7f779effd640 1 -- 172.21.15.102:0/704922282 learned_addr learned my addr 172.21.15.102:0/704922282 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:20.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.618+0000 7f779e7fc640 1 -- 172.21.15.102:0/704922282 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77a0067130 msgr2=0x7f77a011f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:20.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.618+0000 7f779e7fc640 1 --2- 172.21.15.102:0/704922282 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77a0067130 0x7f77a011f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:20.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.618+0000 7f779e7fc640 1 -- 172.21.15.102:0/704922282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77a0098ef0 msgr2=0x7f77a011c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:20.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.618+0000 7f779e7fc640 1 --2- 172.21.15.102:0/704922282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77a0098ef0 0x7f77a011c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:20.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.618+0000 7f779e7fc640 1 -- 172.21.15.102:0/704922282 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7794009450 con 0x7f77a0067a60 2024-09-13T17:42:20.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.618+0000 7f779effd640 1 --2- 172.21.15.102:0/704922282 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77a0067130 0x7f77a011f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:20.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.618+0000 7f779e7fc640 1 --2- 172.21.15.102:0/704922282 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f77a0067a60 0x7f77a011c190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7794009fd0 tx=0x7f77940091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:20.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.619+0000 7f7786ffd640 1 -- 172.21.15.102:0/704922282 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779400c940 con 0x7f77a0067a60 2024-09-13T17:42:20.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.619+0000 7f7786ffd640 1 -- 172.21.15.102:0/704922282 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f779400caa0 con 0x7f77a0067a60 2024-09-13T17:42:20.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.619+0000 7f77a595d640 1 -- 172.21.15.102:0/704922282 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7790009390 con 0x7f77a0067a60 2024-09-13T17:42:20.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.619+0000 7f7786ffd640 1 -- 172.21.15.102:0/704922282 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779400ce40 con 0x7f77a0067a60 2024-09-13T17:42:20.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.619+0000 7f77a595d640 1 -- 172.21.15.102:0/704922282 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a011cc10 con 0x7f77a0067a60 2024-09-13T17:42:20.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.619+0000 7f7786ffd640 1 -- 172.21.15.102:0/704922282 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f7794013680 con 0x7f77a0067a60 2024-09-13T17:42:20.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.620+0000 7f77a595d640 1 -- 172.21.15.102:0/704922282 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77a0064000 con 0x7f77a0067a60 2024-09-13T17:42:20.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.620+0000 7f7786ffd640 1 -- 172.21.15.102:0/704922282 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3660+0+0 (secure 0 0 0) 0x7f77940423c0 con 0x7f77a0067a60 2024-09-13T17:42:20.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.624+0000 7f7786ffd640 1 -- 172.21.15.102:0/704922282 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f7794021020 con 0x7f77a0067a60 2024-09-13T17:42:20.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.808+0000 7f77a595d640 1 -- 172.21.15.102:0/704922282 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f77a011be60 con 0x7f77a0067a60 2024-09-13T17:42:20.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.809+0000 7f7786ffd640 1 -- 172.21.15.102:0/704922282 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v5) v1 ==== 53+0+73 (secure 0 0 0) 0x7f77940236b0 con 0x7f77a0067a60 2024-09-13T17:42:20.811 INFO:teuthology.orchestra.run.smithi102.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T17:42:20.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.811+0000 7f77a595d640 1 -- 172.21.15.102:0/704922282 >> v1:172.21.15.102:6816/34937 conn(0x7f778c031160 legacy=0x7f778c033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:20.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.811+0000 7f77a595d640 1 -- 172.21.15.102:0/704922282 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f77a0067a60 msgr2=0x7f77a011c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:20.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.811+0000 7f77a595d640 1 --2- 172.21.15.102:0/704922282 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f77a0067a60 0x7f77a011c190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7794009fd0 tx=0x7f77940091d0 comp rx=0 tx=0).stop 2024-09-13T17:42:20.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.811+0000 7f77a595d640 1 -- 172.21.15.102:0/704922282 shutdown_connections 2024-09-13T17:42:20.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.811+0000 7f77a595d640 1 --2- 172.21.15.102:0/704922282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77a0098ef0 0x7f77a011c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:20.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.811+0000 7f77a595d640 1 --2- 172.21.15.102:0/704922282 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f77a0067a60 0x7f77a011c190 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:20.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.811+0000 7f77a595d640 1 --2- 172.21.15.102:0/704922282 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f77a0067130 0x7f77a011f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:20.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.811+0000 7f77a595d640 1 -- 172.21.15.102:0/704922282 >> 172.21.15.102:0/704922282 conn(0x7f77a005db50 msgr2=0x7f77a0132520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:20.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.811+0000 7f77a595d640 1 -- 172.21.15.102:0/704922282 shutdown_connections 2024-09-13T17:42:20.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:20.811+0000 7f77a595d640 1 -- 172.21.15.102:0/704922282 wait complete. 2024-09-13T17:42:20.823 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T17:42:20.823 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T17:42:20.823 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi171.front.sepia.ceph.com 2024-09-13T17:42:20.823 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-13T17:42:20.823 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T17:42:20.823 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T17:42:20.823 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T17:42:20.823 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T17:42:20.824 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T17:42:20.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:20.824 DEBUG:teuthology.orchestra.run.smithi171:> ip addr 2024-09-13T17:42:20.844 INFO:teuthology.orchestra.run.smithi171.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T17:42:20.844 INFO:teuthology.orchestra.run.smithi171.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T17:42:20.845 INFO:teuthology.orchestra.run.smithi171.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T17:42:20.845 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:42:20.845 INFO:teuthology.orchestra.run.smithi171.stdout: inet6 ::1/128 scope host 2024-09-13T17:42:20.845 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:42:20.845 INFO:teuthology.orchestra.run.smithi171.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T17:42:20.845 INFO:teuthology.orchestra.run.smithi171.stdout: link/ether 0c:c4:7a:d9:39:a2 brd ff:ff:ff:ff:ff:ff 2024-09-13T17:42:20.845 INFO:teuthology.orchestra.run.smithi171.stdout: altname enp7s0f0 2024-09-13T17:42:20.845 INFO:teuthology.orchestra.run.smithi171.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T17:42:20.846 INFO:teuthology.orchestra.run.smithi171.stdout: link/ether 0c:c4:7a:d9:39:a3 brd ff:ff:ff:ff:ff:ff 2024-09-13T17:42:20.846 INFO:teuthology.orchestra.run.smithi171.stdout: altname enp7s0f1 2024-09-13T17:42:20.846 INFO:teuthology.orchestra.run.smithi171.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T17:42:20.846 INFO:teuthology.orchestra.run.smithi171.stdout: link/ether 0c:c4:7a:88:6f:6c brd ff:ff:ff:ff:ff:ff 2024-09-13T17:42:20.846 INFO:teuthology.orchestra.run.smithi171.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T17:42:20.846 INFO:teuthology.orchestra.run.smithi171.stdout: link/ether 0c:c4:7a:88:6f:6d brd ff:ff:ff:ff:ff:ff 2024-09-13T17:42:20.846 INFO:teuthology.orchestra.run.smithi171.stdout: inet 172.21.15.171/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-13T17:42:20.846 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft 42777sec preferred_lft 42777sec 2024-09-13T17:42:20.846 INFO:teuthology.orchestra.run.smithi171.stdout: inet6 fe80::ec4:7aff:fe88:6f6d/64 scope link noprefixroute 2024-09-13T17:42:20.846 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:42:20.847 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-13T17:42:20.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:20.847 DEBUG:teuthology.orchestra.run.smithi171:> (cd / && exec bash -c ' 2024-09-13T17:42:20.848 DEBUG:teuthology.orchestra.run.smithi171:> set -e 2024-09-13T17:42:20.848 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip link add name ceph-brx type bridge 2024-09-13T17:42:20.848 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip addr flush dev ceph-brx 2024-09-13T17:42:20.848 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip link set ceph-brx up 2024-09-13T17:42:20.848 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-13T17:42:20.848 DEBUG:teuthology.orchestra.run.smithi171:> ') 2024-09-13T17:42:21.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:21.015 DEBUG:teuthology.orchestra.run.smithi171:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T17:42:21.040 INFO:teuthology.orchestra.run.smithi171.stdout:1 2024-09-13T17:42:21.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:21.042 DEBUG:teuthology.orchestra.run.smithi171:> route 2024-09-13T17:42:21.110 INFO:teuthology.orchestra.run.smithi171.stdout:Kernel IP routing table 2024-09-13T17:42:21.111 INFO:teuthology.orchestra.run.smithi171.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T17:42:21.111 INFO:teuthology.orchestra.run.smithi171.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-13T17:42:21.111 INFO:teuthology.orchestra.run.smithi171.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-13T17:42:21.111 INFO:teuthology.orchestra.run.smithi171.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T17:42:21.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:21.112 DEBUG:teuthology.orchestra.run.smithi171:> (cd / && exec bash -c ' 2024-09-13T17:42:21.112 DEBUG:teuthology.orchestra.run.smithi171:> set -e 2024-09-13T17:42:21.112 DEBUG:teuthology.orchestra.run.smithi171:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-13T17:42:21.112 DEBUG:teuthology.orchestra.run.smithi171:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-13T17:42:21.112 DEBUG:teuthology.orchestra.run.smithi171:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-13T17:42:21.112 DEBUG:teuthology.orchestra.run.smithi171:> ') 2024-09-13T17:42:21.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:21.350 DEBUG:teuthology.orchestra.run.smithi171:> ip netns list 2024-09-13T17:42:21.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:21.365 DEBUG:teuthology.orchestra.run.smithi171:> ip netns list-id 2024-09-13T17:42:21.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:21.421 DEBUG:teuthology.orchestra.run.smithi171:> (cd / && exec bash -c ' 2024-09-13T17:42:21.421 DEBUG:teuthology.orchestra.run.smithi171:> set -e 2024-09-13T17:42:21.421 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T17:42:21.421 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-13T17:42:21.422 DEBUG:teuthology.orchestra.run.smithi171:> ') 2024-09-13T17:42:21.502 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-13T17:42:21.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:21.503 DEBUG:teuthology.orchestra.run.smithi171:> (cd / && exec bash -c ' 2024-09-13T17:42:21.503 DEBUG:teuthology.orchestra.run.smithi171:> set -e 2024-09-13T17:42:21.503 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-13T17:42:21.503 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-13T17:42:21.503 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-13T17:42:21.503 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-13T17:42:21.503 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-13T17:42:21.503 DEBUG:teuthology.orchestra.run.smithi171:> ') 2024-09-13T17:42:21.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:21.629 DEBUG:teuthology.orchestra.run.smithi171:> (cd / && exec bash -c ' 2024-09-13T17:42:21.629 DEBUG:teuthology.orchestra.run.smithi171:> set -e 2024-09-13T17:42:21.629 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip link set brx.0 up 2024-09-13T17:42:21.629 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip link set dev brx.0 master ceph-brx 2024-09-13T17:42:21.629 DEBUG:teuthology.orchestra.run.smithi171:> ') 2024-09-13T17:42:21.666 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-13T17:42:21.666 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:42:21.666 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-13T17:42:21.726 INFO:teuthology.orchestra.run.smithi171.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-13T17:42:21.727 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:42:21.727 DEBUG:teuthology.orchestra.run.smithi171:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-13T17:42:21.781 DEBUG:teuthology.orchestra.run.smithi171:> sudo modprobe fuse 2024-09-13T17:42:21.847 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:42:21.847 DEBUG:teuthology.orchestra.run.smithi171:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:42:21.911 INFO:teuthology.orchestra.run.smithi171.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:42:21.913 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:42:21.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:21.913 DEBUG:teuthology.orchestra.run.smithi171:> ls /sys/fs/fuse/connections 2024-09-13T17:42:21.968 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-13T17:42:21.968 DEBUG:teuthology.orchestra.run.smithi171:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-13T17:42:22.011 DEBUG:teuthology.orchestra.run.smithi171:> sudo modprobe fuse 2024-09-13T17:42:22.035 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:42:22.035 DEBUG:teuthology.orchestra.run.smithi171:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:42:22.100 INFO:teuthology.orchestra.run.smithi171.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:42:22.101 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:42:22.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:22.102 DEBUG:teuthology.orchestra.run.smithi171:> ls /sys/fs/fuse/connections 2024-09-13T17:42:22.135 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi171.stderr:2024-09-13T17:42:22.133+0000 7f5cf44c3300 -1 init, newargv = 0x555a12fd7320 newargc=15 2024-09-13T17:42:22.135 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi171.stderr:ceph-fuse[29183]: starting ceph client 2024-09-13T17:42:22.140 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi171.stderr:ceph-fuse[29183]: starting fuse 2024-09-13T17:42:22.150 INFO:teuthology.orchestra.run.smithi171.stdout:43 2024-09-13T17:42:22.150 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-09-13T17:42:22.151 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:42:22.151 DEBUG:teuthology.orchestra.run.smithi171:> import glob 2024-09-13T17:42:22.151 DEBUG:teuthology.orchestra.run.smithi171:> import re 2024-09-13T17:42:22.151 DEBUG:teuthology.orchestra.run.smithi171:> import os 2024-09-13T17:42:22.151 DEBUG:teuthology.orchestra.run.smithi171:> import subprocess 2024-09-13T17:42:22.151 DEBUG:teuthology.orchestra.run.smithi171:> 2024-09-13T17:42:22.152 DEBUG:teuthology.orchestra.run.smithi171:> def _find_admin_socket(client_name): 2024-09-13T17:42:22.152 DEBUG:teuthology.orchestra.run.smithi171:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-13T17:42:22.152 DEBUG:teuthology.orchestra.run.smithi171:> files = glob.glob(asok_path) 2024-09-13T17:42:22.152 DEBUG:teuthology.orchestra.run.smithi171:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-13T17:42:22.152 DEBUG:teuthology.orchestra.run.smithi171:> 2024-09-13T17:42:22.152 DEBUG:teuthology.orchestra.run.smithi171:> # Given a non-glob path, it better be there 2024-09-13T17:42:22.152 DEBUG:teuthology.orchestra.run.smithi171:> if "*" not in asok_path: 2024-09-13T17:42:22.152 DEBUG:teuthology.orchestra.run.smithi171:> assert(len(files) == 1) 2024-09-13T17:42:22.152 DEBUG:teuthology.orchestra.run.smithi171:> return files[0] 2024-09-13T17:42:22.152 DEBUG:teuthology.orchestra.run.smithi171:> 2024-09-13T17:42:22.152 DEBUG:teuthology.orchestra.run.smithi171:> for f in files: 2024-09-13T17:42:22.153 DEBUG:teuthology.orchestra.run.smithi171:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:42:22.153 DEBUG:teuthology.orchestra.run.smithi171:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:42:22.153 DEBUG:teuthology.orchestra.run.smithi171:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:42:22.153 DEBUG:teuthology.orchestra.run.smithi171:> contents = proc_f.read() 2024-09-13T17:42:22.153 DEBUG:teuthology.orchestra.run.smithi171:> if mountpoint in contents: 2024-09-13T17:42:22.153 DEBUG:teuthology.orchestra.run.smithi171:> return f 2024-09-13T17:42:22.153 DEBUG:teuthology.orchestra.run.smithi171:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:42:22.153 DEBUG:teuthology.orchestra.run.smithi171:> 2024-09-13T17:42:22.153 DEBUG:teuthology.orchestra.run.smithi171:> print(_find_admin_socket("client.0")) 2024-09-13T17:42:22.153 DEBUG:teuthology.orchestra.run.smithi171:> ' 2024-09-13T17:42:22.265 INFO:teuthology.orchestra.run.smithi171.stdout:/var/run/ceph/ceph-client.0.29183.asok 2024-09-13T17:42:22.451 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.29183.asok 2024-09-13T17:42:22.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:22.451 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.29183.asok status 2024-09-13T17:42:22.579 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-13T17:42:22.580 INFO:teuthology.orchestra.run.smithi171.stdout: "metadata": { 2024-09-13T17:42:22.580 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_sha1": "bbdadfa31f8fb9afaa389358d138cac49bd69e56", 2024-09-13T17:42:22.580 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_version": "ceph version 17.2.7-1627-gbbdadfa3 (bbdadfa31f8fb9afaa389358d138cac49bd69e56) quincy (stable)", 2024-09-13T17:42:22.580 INFO:teuthology.orchestra.run.smithi171.stdout: "entity_id": "0", 2024-09-13T17:42:22.580 INFO:teuthology.orchestra.run.smithi171.stdout: "hostname": "smithi171", 2024-09-13T17:42:22.580 INFO:teuthology.orchestra.run.smithi171.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-13T17:42:22.580 INFO:teuthology.orchestra.run.smithi171.stdout: "pid": "29183", 2024-09-13T17:42:22.580 INFO:teuthology.orchestra.run.smithi171.stdout: "root": "/" 2024-09-13T17:42:22.580 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-13T17:42:22.580 INFO:teuthology.orchestra.run.smithi171.stdout: "dentry_count": 0, 2024-09-13T17:42:22.581 INFO:teuthology.orchestra.run.smithi171.stdout: "dentry_pinned_count": 0, 2024-09-13T17:42:22.581 INFO:teuthology.orchestra.run.smithi171.stdout: "id": 4531, 2024-09-13T17:42:22.581 INFO:teuthology.orchestra.run.smithi171.stdout: "inst": { 2024-09-13T17:42:22.581 INFO:teuthology.orchestra.run.smithi171.stdout: "name": { 2024-09-13T17:42:22.581 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "client", 2024-09-13T17:42:22.581 INFO:teuthology.orchestra.run.smithi171.stdout: "num": 4531 2024-09-13T17:42:22.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-13T17:42:22.581 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": { 2024-09-13T17:42:22.581 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-13T17:42:22.581 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:0", 2024-09-13T17:42:22.581 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2332732957 2024-09-13T17:42:22.581 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-13T17:42:22.581 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-13T17:42:22.582 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": { 2024-09-13T17:42:22.582 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-13T17:42:22.582 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:0", 2024-09-13T17:42:22.582 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2332732957 2024-09-13T17:42:22.582 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-13T17:42:22.582 INFO:teuthology.orchestra.run.smithi171.stdout: "inst_str": "client.4531 172.21.15.171:0/2332732957", 2024-09-13T17:42:22.582 INFO:teuthology.orchestra.run.smithi171.stdout: "addr_str": "172.21.15.171:0/2332732957", 2024-09-13T17:42:22.582 INFO:teuthology.orchestra.run.smithi171.stdout: "inode_count": 1, 2024-09-13T17:42:22.582 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_epoch": 5, 2024-09-13T17:42:22.582 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_epoch": 21, 2024-09-13T17:42:22.582 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_epoch_barrier": 0, 2024-09-13T17:42:22.582 INFO:teuthology.orchestra.run.smithi171.stdout: "blocklisted": false, 2024-09-13T17:42:22.582 INFO:teuthology.orchestra.run.smithi171.stdout: "fs_name": "cephfs" 2024-09-13T17:42:22.582 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-13T17:42:22.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:22.588 DEBUG:teuthology.orchestra.run.smithi171:> stat --file-system '--printf=%T 2024-09-13T17:42:22.588 DEBUG:teuthology.orchestra.run.smithi171:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-13T17:42:22.604 INFO:teuthology.orchestra.run.smithi171.stdout:fuseblk 2024-09-13T17:42:22.604 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-13T17:42:22.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:22.605 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-13T17:42:22.670 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-13T17:42:22.680 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-13T17:42:22.680 INFO:tasks.ceph_fuse:config is {'client.0': {'mounted': False}} 2024-09-13T17:42:22.680 INFO:tasks.ceph_fuse:client.0 config is {'mounted': False} 2024-09-13T17:42:22.680 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T17:42:22.680 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:42:22.726 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T17:42:22.726 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:42:22.781 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:22.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.881+0000 7ff03da78640 1 Processor -- start 2024-09-13T17:42:22.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.881+0000 7ff03da78640 1 -- start start 2024-09-13T17:42:22.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.881+0000 7ff03da78640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff03812d590 0x7ff03812d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:22.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.881+0000 7ff03da78640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff038067840 0x7ff0381355d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:22.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.881+0000 7ff03da78640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff038099ec0 con 0x7ff03812dea0 2024-09-13T17:42:22.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.881+0000 7ff03da78640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff03812d390 con 0x7ff03812d590 2024-09-13T17:42:22.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.881+0000 7ff03da78640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff03809bc30 con 0x7ff038067840 2024-09-13T17:42:22.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.881+0000 7ff0377fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff03812d590 0x7ff03812d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:22.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.881+0000 7ff036ffd640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7ff03812dea0 0x7ff038067130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:47360/0 (socket says 172.21.15.102:47360) 2024-09-13T17:42:22.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.881+0000 7ff0377fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff03812d590 0x7ff03812d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57974/0 (socket says 172.21.15.102:57974) 2024-09-13T17:42:22.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.881+0000 7ff036ffd640 1 -- 172.21.15.102:0/3452509494 learned_addr learned my addr 172.21.15.102:0/3452509494 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:42:22.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.881+0000 7ff037fff640 1 --2- 172.21.15.102:0/3452509494 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff038067840 0x7ff0381355d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:22.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.882+0000 7ff037fff640 1 -- 172.21.15.102:0/3452509494 >> v2:172.21.15.102:3300/0 conn(0x7ff03812d590 msgr2=0x7ff03812d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:22.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.882+0000 7ff037fff640 1 --2- 172.21.15.102:0/3452509494 >> v2:172.21.15.102:3300/0 conn(0x7ff03812d590 0x7ff03812d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:22.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.882+0000 7ff037fff640 1 -- 172.21.15.102:0/3452509494 >> v1:172.21.15.102:6789/0 conn(0x7ff03812dea0 legacy=0x7ff038067130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:22.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.882+0000 7ff037fff640 1 -- 172.21.15.102:0/3452509494 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff03809aab0 con 0x7ff038067840 2024-09-13T17:42:22.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.882+0000 7ff0367fc640 1 -- 172.21.15.102:0/3452509494 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1483089905 0 0) 0x7ff038099ec0 con 0x7ff03812dea0 2024-09-13T17:42:22.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.882+0000 7ff0377fe640 1 --2- 172.21.15.102:0/3452509494 >> v2:172.21.15.102:3300/0 conn(0x7ff03812d590 0x7ff03812d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:22.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.882+0000 7ff037fff640 1 --2- 172.21.15.102:0/3452509494 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff038067840 0x7ff0381355d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff03000f300 tx=0x7ff03000db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c99625e39acdbd87 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:22.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.882+0000 7ff0367fc640 1 -- 172.21.15.102:0/3452509494 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03000d2e0 con 0x7ff038067840 2024-09-13T17:42:22.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.882+0000 7ff0367fc640 1 -- 172.21.15.102:0/3452509494 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff03000d440 con 0x7ff038067840 2024-09-13T17:42:22.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.882+0000 7ff0367fc640 1 -- 172.21.15.102:0/3452509494 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03000b980 con 0x7ff038067840 2024-09-13T17:42:22.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.883+0000 7ff03da78640 1 -- 172.21.15.102:0/3452509494 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff038067840 msgr2=0x7ff0381355d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:22.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.883+0000 7ff03da78640 1 --2- 172.21.15.102:0/3452509494 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff038067840 0x7ff0381355d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff03000f300 tx=0x7ff03000db90 comp rx=0 tx=0).stop 2024-09-13T17:42:22.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.883+0000 7ff03da78640 1 -- 172.21.15.102:0/3452509494 shutdown_connections 2024-09-13T17:42:22.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.883+0000 7ff03da78640 1 --2- 172.21.15.102:0/3452509494 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff038067840 0x7ff0381355d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:22.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.883+0000 7ff03da78640 1 --2- 172.21.15.102:0/3452509494 >> v2:172.21.15.102:3300/0 conn(0x7ff03812d590 0x7ff03812d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:22.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.883+0000 7ff03da78640 1 -- 172.21.15.102:0/3452509494 >> 172.21.15.102:0/3452509494 conn(0x7ff03805db50 msgr2=0x7ff03805df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:22.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.883+0000 7ff03da78640 1 -- 172.21.15.102:0/3452509494 shutdown_connections 2024-09-13T17:42:22.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.883+0000 7ff03da78640 1 -- 172.21.15.102:0/3452509494 wait complete. 2024-09-13T17:42:22.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.884+0000 7ff03da78640 1 Processor -- start 2024-09-13T17:42:22.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.884+0000 7ff03da78640 1 -- start start 2024-09-13T17:42:22.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.884+0000 7ff03da78640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff038067840 0x7ff03812bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:22.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.884+0000 7ff03da78640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff03812d590 0x7ff03812c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:22.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.884+0000 7ff03da78640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff03812dea0 0x7ff0381190b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:22.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.884+0000 7ff03da78640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03809a220 con 0x7ff03812dea0 2024-09-13T17:42:22.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.884+0000 7ff03da78640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff03811c580 con 0x7ff03812d590 2024-09-13T17:42:22.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.884+0000 7ff03da78640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff03811a520 con 0x7ff038067840 2024-09-13T17:42:22.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.885+0000 7ff0377fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff038067840 0x7ff03812bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:22.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.885+0000 7ff036ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff03812d590 0x7ff03812c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:22.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.885+0000 7ff037fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff03812dea0 0x7ff0381190b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:22.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.885+0000 7ff0377fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff038067840 0x7ff03812bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48840/0 (socket says 172.21.15.102:48840) 2024-09-13T17:42:22.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.885+0000 7ff036ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff03812d590 0x7ff03812c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:39558/0 (socket says 172.21.15.102:39558) 2024-09-13T17:42:22.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.885+0000 7ff0377fe640 1 -- 172.21.15.102:0/84241306 learned_addr learned my addr 172.21.15.102:0/84241306 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:22.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.885+0000 7ff037fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff03812dea0 0x7ff0381190b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57978/0 (socket says 172.21.15.102:57978) 2024-09-13T17:42:22.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.885+0000 7ff037fff640 1 -- 172.21.15.102:0/84241306 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff038067840 msgr2=0x7ff03812bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:22.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.885+0000 7ff037fff640 1 --2- 172.21.15.102:0/84241306 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff038067840 0x7ff03812bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:22.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.885+0000 7ff037fff640 1 -- 172.21.15.102:0/84241306 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff03812d590 msgr2=0x7ff03812c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:22.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.885+0000 7ff037fff640 1 --2- 172.21.15.102:0/84241306 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff03812d590 0x7ff03812c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:22.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.885+0000 7ff037fff640 1 -- 172.21.15.102:0/84241306 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff028009390 con 0x7ff03812dea0 2024-09-13T17:42:22.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.885+0000 7ff036ffd640 1 --2- 172.21.15.102:0/84241306 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff03812d590 0x7ff03812c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:22.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.885+0000 7ff037fff640 1 --2- 172.21.15.102:0/84241306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff03812dea0 0x7ff0381190b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff030035000 tx=0x7ff03003b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:22.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.885+0000 7ff01f7fe640 1 -- 172.21.15.102:0/84241306 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff030041070 con 0x7ff03812dea0 2024-09-13T17:42:22.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.885+0000 7ff01f7fe640 1 -- 172.21.15.102:0/84241306 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff03000d5b0 con 0x7ff03812dea0 2024-09-13T17:42:22.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.885+0000 7ff01f7fe640 1 -- 172.21.15.102:0/84241306 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff030032440 con 0x7ff03812dea0 2024-09-13T17:42:22.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.885+0000 7ff03da78640 1 -- 172.21.15.102:0/84241306 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff03000de30 con 0x7ff03812dea0 2024-09-13T17:42:22.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.885+0000 7ff03da78640 1 -- 172.21.15.102:0/84241306 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff03811d7a0 con 0x7ff03812dea0 2024-09-13T17:42:22.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.886+0000 7ff01f7fe640 1 -- 172.21.15.102:0/84241306 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7ff030045070 con 0x7ff03812dea0 2024-09-13T17:42:22.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.886+0000 7ff01f7fe640 1 -- 172.21.15.102:0/84241306 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3660+0+0 (secure 0 0 0) 0x7ff030066fc0 con 0x7ff03812dea0 2024-09-13T17:42:22.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.886+0000 7ff03da78640 1 -- 172.21.15.102:0/84241306 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0380681e0 con 0x7ff03812dea0 2024-09-13T17:42:22.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:22.891+0000 7ff01f7fe640 1 -- 172.21.15.102:0/84241306 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7ff03004c080 con 0x7ff03812dea0 2024-09-13T17:42:23.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.042+0000 7ff03da78640 1 -- 172.21.15.102:0/84241306 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff038064000 con 0x7ff03812dea0 2024-09-13T17:42:23.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.043+0000 7ff01f7fe640 1 -- 172.21.15.102:0/84241306 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+11330 (secure 0 0 0) 0x7ff030047420 con 0x7ff03812dea0 2024-09-13T17:42:23.045 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:42:23.045 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":21,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:42:20.337430+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:04.672265+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:05.583134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:23.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.044+0000 7ff03da78640 1 -- 172.21.15.102:0/84241306 >> v1:172.21.15.102:6816/34937 conn(0x7ff024031160 legacy=0x7ff024033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:23.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.044+0000 7ff03da78640 1 -- 172.21.15.102:0/84241306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff03812dea0 msgr2=0x7ff0381190b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:23.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.044+0000 7ff03da78640 1 --2- 172.21.15.102:0/84241306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff03812dea0 0x7ff0381190b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff030035000 tx=0x7ff03003b000 comp rx=0 tx=0).stop 2024-09-13T17:42:23.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.044+0000 7ff03da78640 1 -- 172.21.15.102:0/84241306 shutdown_connections 2024-09-13T17:42:23.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.045+0000 7ff03da78640 1 --2- 172.21.15.102:0/84241306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff03812dea0 0x7ff0381190b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.045+0000 7ff03da78640 1 --2- 172.21.15.102:0/84241306 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff03812d590 0x7ff03812c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.045+0000 7ff03da78640 1 --2- 172.21.15.102:0/84241306 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff038067840 0x7ff03812bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.045+0000 7ff03da78640 1 -- 172.21.15.102:0/84241306 >> 172.21.15.102:0/84241306 conn(0x7ff03805db50 msgr2=0x7ff03809e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:23.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.045+0000 7ff03da78640 1 -- 172.21.15.102:0/84241306 shutdown_connections 2024-09-13T17:42:23.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.045+0000 7ff03da78640 1 -- 172.21.15.102:0/84241306 wait complete. 2024-09-13T17:42:23.057 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:54.442612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T17:41:59.076782+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 16, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:42:04.672265+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:42:05.583134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:42:23.058 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:42:23.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.151+0000 7fb2d5b1d640 1 Processor -- start 2024-09-13T17:42:23.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.152+0000 7fb2d5b1d640 1 -- start start 2024-09-13T17:42:23.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.152+0000 7fb2d5b1d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb2d0098ed0 0x7fb2d0131610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:23.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.152+0000 7fb2d5b1d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2d0131b50 0x7fb2d0133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:23.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.152+0000 7fb2d5b1d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2d00999e0 con 0x7fb2d0096a70 2024-09-13T17:42:23.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.152+0000 7fb2d5b1d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2d0069c30 con 0x7fb2d0098ed0 2024-09-13T17:42:23.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.152+0000 7fb2d5b1d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d006ed00 con 0x7fb2d0131b50 2024-09-13T17:42:23.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.152+0000 7fb2c6ffd640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb2d0098ed0 0x7fb2d0131610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:23.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.152+0000 7fb2cffff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2d0131b50 0x7fb2d0133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:23.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.152+0000 7fb2c6ffd640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb2d0098ed0 0x7fb2d0131610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57994/0 (socket says 172.21.15.102:57994) 2024-09-13T17:42:23.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.152+0000 7fb2cf7fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fb2d0096a70 0x7fb2d00987c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:47368/0 (socket says 172.21.15.102:47368) 2024-09-13T17:42:23.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.152+0000 7fb2c6ffd640 1 -- 172.21.15.102:0/2039815582 learned_addr learned my addr 172.21.15.102:0/2039815582 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:23.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.152+0000 7fb2c6ffd640 1 -- 172.21.15.102:0/2039815582 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2d0131b50 msgr2=0x7fb2d0133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:23.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.152+0000 7fb2c6ffd640 1 --2- 172.21.15.102:0/2039815582 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2d0131b50 0x7fb2d0133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.152+0000 7fb2c6ffd640 1 -- 172.21.15.102:0/2039815582 >> v1:172.21.15.102:6789/0 conn(0x7fb2d0096a70 legacy=0x7fb2d00987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:23.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.152+0000 7fb2c6ffd640 1 -- 172.21.15.102:0/2039815582 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d00997a0 con 0x7fb2d0098ed0 2024-09-13T17:42:23.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.152+0000 7fb2cffff640 1 --2- 172.21.15.102:0/2039815582 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2d0131b50 0x7fb2d0133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:23.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.153+0000 7fb2c6ffd640 1 --2- 172.21.15.102:0/2039815582 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2d0098ed0 0x7fb2d0131610 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8009940 tx=0x7fb2b80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0051a715e052f96 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:23.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.153+0000 7fb2ceffd640 1 -- 172.21.15.102:0/2039815582 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b802adb0 con 0x7fb2d0098ed0 2024-09-13T17:42:23.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.153+0000 7fb2ceffd640 1 -- 172.21.15.102:0/2039815582 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b8032440 con 0x7fb2d0098ed0 2024-09-13T17:42:23.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.153+0000 7fb2ceffd640 1 -- 172.21.15.102:0/2039815582 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b8032820 con 0x7fb2d0098ed0 2024-09-13T17:42:23.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.154+0000 7fb2d5b1d640 1 -- 172.21.15.102:0/2039815582 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2d0098ed0 msgr2=0x7fb2d0131610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:23.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.154+0000 7fb2d5b1d640 1 --2- 172.21.15.102:0/2039815582 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2d0098ed0 0x7fb2d0131610 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8009940 tx=0x7fb2b80091d0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.154+0000 7fb2d5b1d640 1 -- 172.21.15.102:0/2039815582 shutdown_connections 2024-09-13T17:42:23.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.154+0000 7fb2d5b1d640 1 --2- 172.21.15.102:0/2039815582 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2d0131b50 0x7fb2d0133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.154+0000 7fb2d5b1d640 1 --2- 172.21.15.102:0/2039815582 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2d0098ed0 0x7fb2d0131610 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.154+0000 7fb2d5b1d640 1 -- 172.21.15.102:0/2039815582 >> 172.21.15.102:0/2039815582 conn(0x7fb2d009e720 msgr2=0x7fb2d009eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:23.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.154+0000 7fb2d5b1d640 1 -- 172.21.15.102:0/2039815582 shutdown_connections 2024-09-13T17:42:23.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.154+0000 7fb2d5b1d640 1 -- 172.21.15.102:0/2039815582 wait complete. 2024-09-13T17:42:23.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.155+0000 7fb2d5b1d640 1 Processor -- start 2024-09-13T17:42:23.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.155+0000 7fb2d5b1d640 1 -- start start 2024-09-13T17:42:23.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.155+0000 7fb2d5b1d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2d0096a70 0x7fb2d013c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:23.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.155+0000 7fb2d5b1d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2d0098ed0 0x7fb2d013cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:23.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.155+0000 7fb2d5b1d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2d0131b50 0x7fb2d013d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:23.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.155+0000 7fb2d5b1d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d01317f0 con 0x7fb2d0098ed0 2024-09-13T17:42:23.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.155+0000 7fb2d5b1d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d006d800 con 0x7fb2d0131b50 2024-09-13T17:42:23.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.155+0000 7fb2d5b1d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d006bc40 con 0x7fb2d0096a70 2024-09-13T17:42:23.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.155+0000 7fb2cf7fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2d0096a70 0x7fb2d013c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:23.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.155+0000 7fb2c6ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2d0098ed0 0x7fb2d013cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:23.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.155+0000 7fb2cf7fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2d0096a70 0x7fb2d013c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48856/0 (socket says 172.21.15.102:48856) 2024-09-13T17:42:23.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.155+0000 7fb2c6ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2d0098ed0 0x7fb2d013cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58008/0 (socket says 172.21.15.102:58008) 2024-09-13T17:42:23.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.155+0000 7fb2cf7fe640 1 -- 172.21.15.102:0/1072680977 learned_addr learned my addr 172.21.15.102:0/1072680977 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:23.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.155+0000 7fb2cffff640 1 --2- 172.21.15.102:0/1072680977 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2d0131b50 0x7fb2d013d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:23.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.156+0000 7fb2cf7fe640 1 -- 172.21.15.102:0/1072680977 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2d0131b50 msgr2=0x7fb2d013d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:23.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.156+0000 7fb2cf7fe640 1 --2- 172.21.15.102:0/1072680977 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2d0131b50 0x7fb2d013d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.156+0000 7fb2cf7fe640 1 -- 172.21.15.102:0/1072680977 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2d0098ed0 msgr2=0x7fb2d013cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:23.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.156+0000 7fb2cf7fe640 1 --2- 172.21.15.102:0/1072680977 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2d0098ed0 0x7fb2d013cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.156+0000 7fb2cf7fe640 1 -- 172.21.15.102:0/1072680977 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c8009a20 con 0x7fb2d0096a70 2024-09-13T17:42:23.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.156+0000 7fb2c6ffd640 1 --2- 172.21.15.102:0/1072680977 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2d0098ed0 0x7fb2d013cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:23.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.156+0000 7fb2cf7fe640 1 --2- 172.21.15.102:0/1072680977 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2d0096a70 0x7fb2d013c890 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000c4e0 tx=0x7fb2c0007ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:23.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.156+0000 7fb2c7fff640 1 -- 172.21.15.102:0/1072680977 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c00072e0 con 0x7fb2d0096a70 2024-09-13T17:42:23.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.156+0000 7fb2c7fff640 1 -- 172.21.15.102:0/1072680977 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c0007480 con 0x7fb2d0096a70 2024-09-13T17:42:23.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.156+0000 7fb2d5b1d640 1 -- 172.21.15.102:0/1072680977 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2b8009450 con 0x7fb2d0096a70 2024-09-13T17:42:23.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.156+0000 7fb2c7fff640 1 -- 172.21.15.102:0/1072680977 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c00047a0 con 0x7fb2d0096a70 2024-09-13T17:42:23.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.156+0000 7fb2d5b1d640 1 -- 172.21.15.102:0/1072680977 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d006c3c0 con 0x7fb2d0096a70 2024-09-13T17:42:23.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.157+0000 7fb2c7fff640 1 -- 172.21.15.102:0/1072680977 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fb2c0004bd0 con 0x7fb2d0096a70 2024-09-13T17:42:23.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.157+0000 7fb2c7fff640 1 -- 172.21.15.102:0/1072680977 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3660+0+0 (secure 0 0 0) 0x7fb2c0041850 con 0x7fb2d0096a70 2024-09-13T17:42:23.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.157+0000 7fb2d5b1d640 1 -- 172.21.15.102:0/1072680977 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d0064000 con 0x7fb2d0096a70 2024-09-13T17:42:23.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.161+0000 7fb2c7fff640 1 -- 172.21.15.102:0/1072680977 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fb2c00125b0 con 0x7fb2d0096a70 2024-09-13T17:42:23.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.311+0000 7fb2d5b1d640 1 -- 172.21.15.102:0/1072680977 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb2d006c5b0 con 0x7fb2d0096a70 2024-09-13T17:42:23.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.312+0000 7fb2c7fff640 1 -- 172.21.15.102:0/1072680977 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb2c0010300 con 0x7fb2d0096a70 2024-09-13T17:42:23.314 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-09-13T17:42:23.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.314+0000 7fb2d5b1d640 1 -- 172.21.15.102:0/1072680977 >> v1:172.21.15.102:6816/34937 conn(0x7fb2b0031160 legacy=0x7fb2b0033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:23.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.314+0000 7fb2d5b1d640 1 -- 172.21.15.102:0/1072680977 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2d0096a70 msgr2=0x7fb2d013c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:23.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.314+0000 7fb2d5b1d640 1 --2- 172.21.15.102:0/1072680977 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2d0096a70 0x7fb2d013c890 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000c4e0 tx=0x7fb2c0007ee0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.314+0000 7fb2d5b1d640 1 -- 172.21.15.102:0/1072680977 shutdown_connections 2024-09-13T17:42:23.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.314+0000 7fb2d5b1d640 1 --2- 172.21.15.102:0/1072680977 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb2d0131b50 0x7fb2d013d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.314+0000 7fb2d5b1d640 1 --2- 172.21.15.102:0/1072680977 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2d0098ed0 0x7fb2d013cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.314+0000 7fb2d5b1d640 1 --2- 172.21.15.102:0/1072680977 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb2d0096a70 0x7fb2d013c890 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.314+0000 7fb2d5b1d640 1 -- 172.21.15.102:0/1072680977 >> 172.21.15.102:0/1072680977 conn(0x7fb2d009e720 msgr2=0x7fb2d0133480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:23.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.314+0000 7fb2d5b1d640 1 -- 172.21.15.102:0/1072680977 shutdown_connections 2024-09-13T17:42:23.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.314+0000 7fb2d5b1d640 1 -- 172.21.15.102:0/1072680977 wait complete. 2024-09-13T17:42:23.327 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-13T17:42:23.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.422+0000 7f794db9d640 1 Processor -- start 2024-09-13T17:42:23.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.422+0000 7f794db9d640 1 -- start start 2024-09-13T17:42:23.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.423+0000 7f794db9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f79480689d0 0x7f794809d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:23.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.423+0000 7f794db9d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f794809dd70 0x7f794812f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:23.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.423+0000 7f794db9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f794809e160 0x7f794812fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:23.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.423+0000 7f794db9d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7948068da0 con 0x7f794809dd70 2024-09-13T17:42:23.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.423+0000 7f794db9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7948096db0 con 0x7f794809e160 2024-09-13T17:42:23.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.423+0000 7f794db9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7948097730 con 0x7f79480689d0 2024-09-13T17:42:23.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.423+0000 7f7946ffd640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f794809dd70 0x7f794812f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:23.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.423+0000 7f79477fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f79480689d0 0x7f794809d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:23.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.423+0000 7f7947fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f794809e160 0x7f794812fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:23.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.423+0000 7f7946ffd640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f794809dd70 0x7f794812f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58022/0 (socket says 172.21.15.102:58022) 2024-09-13T17:42:23.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.423+0000 7f79477fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f79480689d0 0x7f794809d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48858/0 (socket says 172.21.15.102:48858) 2024-09-13T17:42:23.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.423+0000 7f7947fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f794809e160 0x7f794812fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:39580/0 (socket says 172.21.15.102:39580) 2024-09-13T17:42:23.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.423+0000 7f7946ffd640 1 -- 172.21.15.102:0/5963695 learned_addr learned my addr 172.21.15.102:0/5963695 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:23.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.423+0000 7f7946ffd640 1 -- 172.21.15.102:0/5963695 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f79480689d0 msgr2=0x7f794809d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:23.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.423+0000 7f7946ffd640 1 --2- 172.21.15.102:0/5963695 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f79480689d0 0x7f794809d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.423+0000 7f7946ffd640 1 -- 172.21.15.102:0/5963695 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f794809e160 msgr2=0x7f794812fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:23.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.423+0000 7f7946ffd640 1 --2- 172.21.15.102:0/5963695 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f794809e160 0x7f794812fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.423+0000 7f7946ffd640 1 -- 172.21.15.102:0/5963695 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7948096b10 con 0x7f794809dd70 2024-09-13T17:42:23.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.423+0000 7f7947fff640 1 --2- 172.21.15.102:0/5963695 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f794809e160 0x7f794812fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:23.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.423+0000 7f79477fe640 1 --2- 172.21.15.102:0/5963695 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f79480689d0 0x7f794809d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:23.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.424+0000 7f7946ffd640 1 --2- 172.21.15.102:0/5963695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f794809dd70 0x7f794812f610 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f793c009940 tx=0x7f793c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27a7cdca54fa117 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:23.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.424+0000 7f79467fc640 1 -- 172.21.15.102:0/5963695 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c02ac70 con 0x7f794809dd70 2024-09-13T17:42:23.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.424+0000 7f79467fc640 1 -- 172.21.15.102:0/5963695 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f793c02add0 con 0x7f794809dd70 2024-09-13T17:42:23.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.424+0000 7f79467fc640 1 -- 172.21.15.102:0/5963695 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c032670 con 0x7f794809dd70 2024-09-13T17:42:23.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.425+0000 7f794db9d640 1 -- 172.21.15.102:0/5963695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f794809dd70 msgr2=0x7f794812f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:23.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.425+0000 7f794db9d640 1 --2- 172.21.15.102:0/5963695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f794809dd70 0x7f794812f610 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f793c009940 tx=0x7f793c0091d0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.425+0000 7f794db9d640 1 -- 172.21.15.102:0/5963695 shutdown_connections 2024-09-13T17:42:23.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.425+0000 7f794db9d640 1 --2- 172.21.15.102:0/5963695 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f794809e160 0x7f794812fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.425+0000 7f794db9d640 1 --2- 172.21.15.102:0/5963695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f794809dd70 0x7f794812f610 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.425+0000 7f794db9d640 1 --2- 172.21.15.102:0/5963695 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f79480689d0 0x7f794809d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.425+0000 7f794db9d640 1 -- 172.21.15.102:0/5963695 >> 172.21.15.102:0/5963695 conn(0x7f794806f320 msgr2=0x7f79480a2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:23.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.425+0000 7f794db9d640 1 -- 172.21.15.102:0/5963695 shutdown_connections 2024-09-13T17:42:23.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.425+0000 7f794db9d640 1 -- 172.21.15.102:0/5963695 wait complete. 2024-09-13T17:42:23.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.426+0000 7f794db9d640 1 Processor -- start 2024-09-13T17:42:23.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.426+0000 7f794db9d640 1 -- start start 2024-09-13T17:42:23.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.426+0000 7f794db9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f79480689d0 0x7f7948121ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:23.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.426+0000 7f794db9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f794809dd70 0x7f794811db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:23.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.426+0000 7f794db9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f794809e160 0x7f794811e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:23.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.426+0000 7f794db9d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7948121010 con 0x7f794809e160 2024-09-13T17:42:23.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.426+0000 7f794db9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79481215b0 con 0x7f79480689d0 2024-09-13T17:42:23.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.426+0000 7f794db9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7948097ce0 con 0x7f794809dd70 2024-09-13T17:42:23.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.426+0000 7f79477fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f79480689d0 0x7f7948121ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:23.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.426+0000 7f79477fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f79480689d0 0x7f7948121ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:39596/0 (socket says 172.21.15.102:39596) 2024-09-13T17:42:23.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.426+0000 7f79477fe640 1 -- 172.21.15.102:0/2417840432 learned_addr learned my addr 172.21.15.102:0/2417840432 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:23.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.426+0000 7f7947fff640 1 --2- 172.21.15.102:0/2417840432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f794809e160 0x7f794811e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:23.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.426+0000 7f7946ffd640 1 --2- 172.21.15.102:0/2417840432 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f794809dd70 0x7f794811db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:23.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.426+0000 7f79477fe640 1 -- 172.21.15.102:0/2417840432 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f794809dd70 msgr2=0x7f794811db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:23.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.426+0000 7f79477fe640 1 --2- 172.21.15.102:0/2417840432 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f794809dd70 0x7f794811db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.426+0000 7f79477fe640 1 -- 172.21.15.102:0/2417840432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f794809e160 msgr2=0x7f794811e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:23.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.426+0000 7f79477fe640 1 --2- 172.21.15.102:0/2417840432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f794809e160 0x7f794811e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.426+0000 7f79477fe640 1 -- 172.21.15.102:0/2417840432 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7938009a20 con 0x7f79480689d0 2024-09-13T17:42:23.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.426+0000 7f7947fff640 1 --2- 172.21.15.102:0/2417840432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f794809e160 0x7f794811e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:23.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.427+0000 7f79477fe640 1 --2- 172.21.15.102:0/2417840432 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f79480689d0 0x7f7948121ab0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f793800c0c0 tx=0x7f793800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:23.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.427+0000 7f792f7fe640 1 -- 172.21.15.102:0/2417840432 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793800cde0 con 0x7f79480689d0 2024-09-13T17:42:23.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.427+0000 7f792f7fe640 1 -- 172.21.15.102:0/2417840432 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f793800f040 con 0x7f79480689d0 2024-09-13T17:42:23.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.427+0000 7f794db9d640 1 -- 172.21.15.102:0/2417840432 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7940009390 con 0x7f79480689d0 2024-09-13T17:42:23.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.427+0000 7f792f7fe640 1 -- 172.21.15.102:0/2417840432 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7938014690 con 0x7f79480689d0 2024-09-13T17:42:23.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.427+0000 7f794db9d640 1 -- 172.21.15.102:0/2417840432 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f793c009450 con 0x7f79480689d0 2024-09-13T17:42:23.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.427+0000 7f792f7fe640 1 -- 172.21.15.102:0/2417840432 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f7938014a00 con 0x7f79480689d0 2024-09-13T17:42:23.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.428+0000 7f792f7fe640 1 -- 172.21.15.102:0/2417840432 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3660+0+0 (secure 0 0 0) 0x7f7938042f00 con 0x7f79480689d0 2024-09-13T17:42:23.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.428+0000 7f794db9d640 1 -- 172.21.15.102:0/2417840432 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f794811f870 con 0x7f79480689d0 2024-09-13T17:42:23.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.432+0000 7f792f7fe640 1 -- 172.21.15.102:0/2417840432 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f7938024530 con 0x7f79480689d0 2024-09-13T17:42:23.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.583+0000 7f794db9d640 1 -- 172.21.15.102:0/2417840432 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7948064000 con 0x7f79480689d0 2024-09-13T17:42:23.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.584+0000 7f792f7fe640 1 -- 172.21.15.102:0/2417840432 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f793802b0a0 con 0x7f79480689d0 2024-09-13T17:42:23.586 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-09-13T17:42:23.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.585+0000 7f794db9d640 1 -- 172.21.15.102:0/2417840432 >> v1:172.21.15.102:6816/34937 conn(0x7f7934031160 legacy=0x7f7934033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:23.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.585+0000 7f794db9d640 1 -- 172.21.15.102:0/2417840432 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f79480689d0 msgr2=0x7f7948121ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:23.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.585+0000 7f794db9d640 1 --2- 172.21.15.102:0/2417840432 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f79480689d0 0x7f7948121ab0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f793800c0c0 tx=0x7f793800b040 comp rx=0 tx=0).stop 2024-09-13T17:42:23.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.585+0000 7f794db9d640 1 -- 172.21.15.102:0/2417840432 shutdown_connections 2024-09-13T17:42:23.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.585+0000 7f794db9d640 1 --2- 172.21.15.102:0/2417840432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f794809e160 0x7f794811e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.585+0000 7f794db9d640 1 --2- 172.21.15.102:0/2417840432 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f794809dd70 0x7f794811db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.585+0000 7f794db9d640 1 --2- 172.21.15.102:0/2417840432 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f79480689d0 0x7f7948121ab0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.585+0000 7f794db9d640 1 -- 172.21.15.102:0/2417840432 >> 172.21.15.102:0/2417840432 conn(0x7f794806f320 msgr2=0x7f794812f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:23.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.585+0000 7f794db9d640 1 -- 172.21.15.102:0/2417840432 shutdown_connections 2024-09-13T17:42:23.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.586+0000 7f794db9d640 1 -- 172.21.15.102:0/2417840432 wait complete. 2024-09-13T17:42:23.598 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:42:23.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.693+0000 7f23ece6f640 1 Processor -- start 2024-09-13T17:42:23.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.693+0000 7f23ece6f640 1 -- start start 2024-09-13T17:42:23.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.693+0000 7f23ece6f640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f23e809a620 0x7f23e8096c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:23.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.693+0000 7f23ece6f640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f23e8097170 0x7f23e8097540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:23.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.693+0000 7f23ece6f640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f23e812d180 0x7f23e812d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:23.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.693+0000 7f23ece6f640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23e809a9f0 con 0x7f23e812d180 2024-09-13T17:42:23.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.693+0000 7f23ece6f640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e80982a0 con 0x7f23e809a620 2024-09-13T17:42:23.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.693+0000 7f23ece6f640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f23e8098040 con 0x7f23e8097170 2024-09-13T17:42:23.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.693+0000 7f23dffff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f23e8097170 0x7f23e8097540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:23.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.693+0000 7f23e6d9d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f23e812d180 0x7f23e812d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:23.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.693+0000 7f23e659c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f23e809a620 0x7f23e8096c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:23.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.693+0000 7f23dffff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f23e8097170 0x7f23e8097540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48872/0 (socket says 172.21.15.102:48872) 2024-09-13T17:42:23.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.693+0000 7f23e659c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f23e809a620 0x7f23e8096c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:39608/0 (socket says 172.21.15.102:39608) 2024-09-13T17:42:23.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.693+0000 7f23dffff640 1 -- 172.21.15.102:0/2575618775 learned_addr learned my addr 172.21.15.102:0/2575618775 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:23.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.694+0000 7f23e6d9d640 1 -- 172.21.15.102:0/2575618775 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f23e8097170 msgr2=0x7f23e8097540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:23.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.694+0000 7f23e6d9d640 1 --2- 172.21.15.102:0/2575618775 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f23e8097170 0x7f23e8097540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.694+0000 7f23e6d9d640 1 -- 172.21.15.102:0/2575618775 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f23e809a620 msgr2=0x7f23e8096c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:23.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.694+0000 7f23e6d9d640 1 --2- 172.21.15.102:0/2575618775 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f23e809a620 0x7f23e8096c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.694+0000 7f23e6d9d640 1 -- 172.21.15.102:0/2575618775 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e809ad50 con 0x7f23e812d180 2024-09-13T17:42:23.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.694+0000 7f23e6d9d640 1 --2- 172.21.15.102:0/2575618775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f23e812d180 0x7f23e812d5c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f23e0009e60 tx=0x7f23e0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f598c07c2c768c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:23.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.694+0000 7f23e5d9b640 1 -- 172.21.15.102:0/2575618775 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e0007550 con 0x7f23e812d180 2024-09-13T17:42:23.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.694+0000 7f23e5d9b640 1 -- 172.21.15.102:0/2575618775 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23e002d050 con 0x7f23e812d180 2024-09-13T17:42:23.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.694+0000 7f23e5d9b640 1 -- 172.21.15.102:0/2575618775 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e00078e0 con 0x7f23e812d180 2024-09-13T17:42:23.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.695+0000 7f23ece6f640 1 -- 172.21.15.102:0/2575618775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f23e812d180 msgr2=0x7f23e812d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:23.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.695+0000 7f23ece6f640 1 --2- 172.21.15.102:0/2575618775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f23e812d180 0x7f23e812d5c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f23e0009e60 tx=0x7f23e0009780 comp rx=0 tx=0).stop 2024-09-13T17:42:23.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.695+0000 7f23ece6f640 1 -- 172.21.15.102:0/2575618775 shutdown_connections 2024-09-13T17:42:23.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.695+0000 7f23ece6f640 1 --2- 172.21.15.102:0/2575618775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f23e812d180 0x7f23e812d5c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.695+0000 7f23ece6f640 1 --2- 172.21.15.102:0/2575618775 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f23e8097170 0x7f23e8097540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.695+0000 7f23ece6f640 1 --2- 172.21.15.102:0/2575618775 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f23e809a620 0x7f23e8096c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.695+0000 7f23ece6f640 1 -- 172.21.15.102:0/2575618775 >> 172.21.15.102:0/2575618775 conn(0x7f23e809e6b0 msgr2=0x7f23e809eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:23.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.695+0000 7f23ece6f640 1 -- 172.21.15.102:0/2575618775 shutdown_connections 2024-09-13T17:42:23.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.695+0000 7f23ece6f640 1 -- 172.21.15.102:0/2575618775 wait complete. 2024-09-13T17:42:23.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.696+0000 7f23ece6f640 1 Processor -- start 2024-09-13T17:42:23.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.696+0000 7f23ece6f640 1 -- start start 2024-09-13T17:42:23.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.696+0000 7f23ece6f640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f23e8097170 0x7f23e8098440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:23.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.696+0000 7f23ece6f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f23e809a620 0x7f23e8098980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:23.703 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.696+0000 7f23ece6f640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f23e812d180 0x7f23e8098ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:23.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.696+0000 7f23ece6f640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e812cf30 con 0x7f23e809a620 2024-09-13T17:42:23.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.696+0000 7f23ece6f640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e8069d20 con 0x7f23e8097170 2024-09-13T17:42:23.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.696+0000 7f23ece6f640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f23e8099fd0 con 0x7f23e812d180 2024-09-13T17:42:23.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.696+0000 7f23e659c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f23e8097170 0x7f23e8098440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:23.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.696+0000 7f23e659c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f23e8097170 0x7f23e8098440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:39618/0 (socket says 172.21.15.102:39618) 2024-09-13T17:42:23.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.696+0000 7f23dffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f23e809a620 0x7f23e8098980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:23.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.696+0000 7f23e659c640 1 -- 172.21.15.102:0/2246728610 learned_addr learned my addr 172.21.15.102:0/2246728610 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:23.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.696+0000 7f23e6d9d640 1 --2- 172.21.15.102:0/2246728610 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f23e812d180 0x7f23e8098ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:23.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.697+0000 7f23e659c640 1 -- 172.21.15.102:0/2246728610 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f23e812d180 msgr2=0x7f23e8098ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:23.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.697+0000 7f23e659c640 1 --2- 172.21.15.102:0/2246728610 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f23e812d180 0x7f23e8098ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.697+0000 7f23e659c640 1 -- 172.21.15.102:0/2246728610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f23e809a620 msgr2=0x7f23e8098980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:23.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.697+0000 7f23e659c640 1 --2- 172.21.15.102:0/2246728610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f23e809a620 0x7f23e8098980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.697+0000 7f23e659c640 1 -- 172.21.15.102:0/2246728610 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e0009a20 con 0x7f23e8097170 2024-09-13T17:42:23.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.697+0000 7f23dffff640 1 --2- 172.21.15.102:0/2246728610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f23e809a620 0x7f23e8098980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:23.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.697+0000 7f23e659c640 1 --2- 172.21.15.102:0/2246728610 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f23e8097170 0x7f23e8098440 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f23d800ea50 tx=0x7f23d80090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:23.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.697+0000 7f23de7fc640 1 -- 172.21.15.102:0/2246728610 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d800ca10 con 0x7f23e8097170 2024-09-13T17:42:23.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.697+0000 7f23de7fc640 1 -- 172.21.15.102:0/2246728610 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d800cb70 con 0x7f23e8097170 2024-09-13T17:42:23.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.697+0000 7f23de7fc640 1 -- 172.21.15.102:0/2246728610 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d8013470 con 0x7f23e8097170 2024-09-13T17:42:23.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.697+0000 7f23ece6f640 1 -- 172.21.15.102:0/2246728610 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e81210f0 con 0x7f23e8097170 2024-09-13T17:42:23.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.697+0000 7f23ece6f640 1 -- 172.21.15.102:0/2246728610 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e81213c0 con 0x7f23e8097170 2024-09-13T17:42:23.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.698+0000 7f23de7fc640 1 -- 172.21.15.102:0/2246728610 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f23d8013860 con 0x7f23e8097170 2024-09-13T17:42:23.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.698+0000 7f23de7fc640 1 -- 172.21.15.102:0/2246728610 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3660+0+0 (secure 0 0 0) 0x7f23d80426f0 con 0x7f23e8097170 2024-09-13T17:42:23.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.698+0000 7f23ece6f640 1 -- 172.21.15.102:0/2246728610 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23e8064000 con 0x7f23e8097170 2024-09-13T17:42:23.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.702+0000 7f23de7fc640 1 -- 172.21.15.102:0/2246728610 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f23d801bb90 con 0x7f23e8097170 2024-09-13T17:42:23.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.854+0000 7f23ece6f640 1 -- 172.21.15.102:0/2246728610 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f23e8120d20 con 0x7f23e8097170 2024-09-13T17:42:23.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.854+0000 7f23de7fc640 1 -- 172.21.15.102:0/2246728610 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f23d802a0a0 con 0x7f23e8097170 2024-09-13T17:42:23.856 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-09-13T17:42:23.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.856+0000 7f23ece6f640 1 -- 172.21.15.102:0/2246728610 >> v1:172.21.15.102:6816/34937 conn(0x7f23c8031160 legacy=0x7f23c8033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:23.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.856+0000 7f23ece6f640 1 -- 172.21.15.102:0/2246728610 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f23e8097170 msgr2=0x7f23e8098440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:23.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.856+0000 7f23ece6f640 1 --2- 172.21.15.102:0/2246728610 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f23e8097170 0x7f23e8098440 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f23d800ea50 tx=0x7f23d80090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.856+0000 7f23ece6f640 1 -- 172.21.15.102:0/2246728610 shutdown_connections 2024-09-13T17:42:23.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.856+0000 7f23ece6f640 1 --2- 172.21.15.102:0/2246728610 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f23e812d180 0x7f23e8098ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.856+0000 7f23ece6f640 1 --2- 172.21.15.102:0/2246728610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f23e809a620 0x7f23e8098980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.856+0000 7f23ece6f640 1 --2- 172.21.15.102:0/2246728610 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f23e8097170 0x7f23e8098440 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.856+0000 7f23ece6f640 1 -- 172.21.15.102:0/2246728610 >> 172.21.15.102:0/2246728610 conn(0x7f23e809e6b0 msgr2=0x7f23e806d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:23.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.856+0000 7f23ece6f640 1 -- 172.21.15.102:0/2246728610 shutdown_connections 2024-09-13T17:42:23.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.856+0000 7f23ece6f640 1 -- 172.21.15.102:0/2246728610 wait complete. 2024-09-13T17:42:23.869 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:42:23.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.963+0000 7f65b94e8640 1 Processor -- start 2024-09-13T17:42:23.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.963+0000 7f65b94e8640 1 -- start start 2024-09-13T17:42:23.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.963+0000 7f65b94e8640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65b4067130 0x7f65b409db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:23.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.963+0000 7f65b94e8640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65b409d6c0 0x7f65b409e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:23.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.963+0000 7f65b94e8640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65b409f5d0 con 0x7f65b412f3f0 2024-09-13T17:42:23.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.963+0000 7f65b94e8640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65b4097800 con 0x7f65b409d6c0 2024-09-13T17:42:23.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.963+0000 7f65b94e8640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f65b4096e80 con 0x7f65b4067130 2024-09-13T17:42:23.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.963+0000 7f65abfff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65b409d6c0 0x7f65b409e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:23.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.963+0000 7f65b37fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f65b412f3f0 0x7f65b412f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:47376/0 (socket says 172.21.15.102:47376) 2024-09-13T17:42:23.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.964+0000 7f65b37fe640 1 -- 172.21.15.102:0/3071383778 learned_addr learned my addr 172.21.15.102:0/3071383778 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:42:23.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.964+0000 7f65b2ffd640 1 --2- 172.21.15.102:0/3071383778 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65b4067130 0x7f65b409db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:23.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.964+0000 7f65abfff640 1 -- 172.21.15.102:0/3071383778 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65b4067130 msgr2=0x7f65b409db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:23.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.964+0000 7f65b27fc640 1 -- 172.21.15.102:0/3071383778 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 835506930 0 0) 0x7f65b409f5d0 con 0x7f65b412f3f0 2024-09-13T17:42:23.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.964+0000 7f65abfff640 1 --2- 172.21.15.102:0/3071383778 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65b4067130 0x7f65b409db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.964+0000 7f65abfff640 1 -- 172.21.15.102:0/3071383778 >> v1:172.21.15.102:6789/0 conn(0x7f65b412f3f0 legacy=0x7f65b412f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:23.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.964+0000 7f65abfff640 1 -- 172.21.15.102:0/3071383778 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b409e5d0 con 0x7f65b409d6c0 2024-09-13T17:42:23.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.964+0000 7f65b2ffd640 1 --2- 172.21.15.102:0/3071383778 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65b4067130 0x7f65b409db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:23.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.964+0000 7f65abfff640 1 --2- 172.21.15.102:0/3071383778 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65b409d6c0 0x7f65b409e090 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f659c009880 tx=0x7f659c0090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abdb809e0046c525 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:23.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.965+0000 7f65b27fc640 1 -- 172.21.15.102:0/3071383778 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c02ace0 con 0x7f65b409d6c0 2024-09-13T17:42:23.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.965+0000 7f65b27fc640 1 -- 172.21.15.102:0/3071383778 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659c02ae40 con 0x7f65b409d6c0 2024-09-13T17:42:23.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.965+0000 7f65b27fc640 1 -- 172.21.15.102:0/3071383778 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c032670 con 0x7f65b409d6c0 2024-09-13T17:42:23.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.965+0000 7f65b94e8640 1 -- 172.21.15.102:0/3071383778 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65b409d6c0 msgr2=0x7f65b409e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:23.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.965+0000 7f65b94e8640 1 --2- 172.21.15.102:0/3071383778 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65b409d6c0 0x7f65b409e090 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f659c009880 tx=0x7f659c0090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.965+0000 7f65b94e8640 1 -- 172.21.15.102:0/3071383778 shutdown_connections 2024-09-13T17:42:23.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.965+0000 7f65b94e8640 1 --2- 172.21.15.102:0/3071383778 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65b409d6c0 0x7f65b409e090 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.965+0000 7f65b94e8640 1 --2- 172.21.15.102:0/3071383778 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65b4067130 0x7f65b409db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.965+0000 7f65b94e8640 1 -- 172.21.15.102:0/3071383778 >> 172.21.15.102:0/3071383778 conn(0x7f65b40a0420 msgr2=0x7f65b405db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:23.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.965+0000 7f65b94e8640 1 -- 172.21.15.102:0/3071383778 shutdown_connections 2024-09-13T17:42:23.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.965+0000 7f65b94e8640 1 -- 172.21.15.102:0/3071383778 wait complete. 2024-09-13T17:42:23.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.966+0000 7f65b94e8640 1 Processor -- start 2024-09-13T17:42:23.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.966+0000 7f65b94e8640 1 -- start start 2024-09-13T17:42:23.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.966+0000 7f65b94e8640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65b4067130 0x7f65b411bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:23.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.966+0000 7f65b94e8640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65b409d6c0 0x7f65b411c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:23.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.966+0000 7f65b94e8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65b412f3f0 0x7f65b411c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:23.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.966+0000 7f65b94e8640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b411db20 con 0x7f65b412f3f0 2024-09-13T17:42:23.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.966+0000 7f65b94e8640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65b409ea90 con 0x7f65b4067130 2024-09-13T17:42:23.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.966+0000 7f65b94e8640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f65b409a900 con 0x7f65b409d6c0 2024-09-13T17:42:23.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.967+0000 7f65b2ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65b4067130 0x7f65b411bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:23.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.967+0000 7f65abfff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65b409d6c0 0x7f65b411c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:23.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.967+0000 7f65b2ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65b4067130 0x7f65b411bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:39640/0 (socket says 172.21.15.102:39640) 2024-09-13T17:42:23.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.967+0000 7f65b2ffd640 1 -- 172.21.15.102:0/3105481865 learned_addr learned my addr 172.21.15.102:0/3105481865 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:23.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.967+0000 7f65b37fe640 1 --2- 172.21.15.102:0/3105481865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65b412f3f0 0x7f65b411c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:23.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.967+0000 7f65abfff640 1 -- 172.21.15.102:0/3105481865 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65b4067130 msgr2=0x7f65b411bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:23.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.967+0000 7f65abfff640 1 --2- 172.21.15.102:0/3105481865 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65b4067130 0x7f65b411bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.967+0000 7f65abfff640 1 -- 172.21.15.102:0/3105481865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65b412f3f0 msgr2=0x7f65b411c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:23.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.967+0000 7f65abfff640 1 --2- 172.21.15.102:0/3105481865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65b412f3f0 0x7f65b411c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:23.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.967+0000 7f65abfff640 1 -- 172.21.15.102:0/3105481865 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a4009390 con 0x7f65b409d6c0 2024-09-13T17:42:23.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.967+0000 7f65b2ffd640 1 --2- 172.21.15.102:0/3105481865 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65b4067130 0x7f65b411bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:23.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.967+0000 7f65abfff640 1 --2- 172.21.15.102:0/3105481865 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65b409d6c0 0x7f65b411c2d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f659c031000 tx=0x7f659c032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:23.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.967+0000 7f65aaffd640 1 -- 172.21.15.102:0/3105481865 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c031700 con 0x7f65b409d6c0 2024-09-13T17:42:23.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.967+0000 7f65aaffd640 1 -- 172.21.15.102:0/3105481865 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659c031860 con 0x7f65b409d6c0 2024-09-13T17:42:23.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.967+0000 7f65aaffd640 1 -- 172.21.15.102:0/3105481865 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c031c20 con 0x7f65b409d6c0 2024-09-13T17:42:23.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.967+0000 7f65b94e8640 1 -- 172.21.15.102:0/3105481865 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f659c009390 con 0x7f65b409d6c0 2024-09-13T17:42:23.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.967+0000 7f65b94e8640 1 -- 172.21.15.102:0/3105481865 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65b411f900 con 0x7f65b409d6c0 2024-09-13T17:42:23.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.968+0000 7f65aaffd640 1 -- 172.21.15.102:0/3105481865 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f659c039040 con 0x7f65b409d6c0 2024-09-13T17:42:23.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.968+0000 7f65aaffd640 1 -- 172.21.15.102:0/3105481865 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3660+0+0 (secure 0 0 0) 0x7f659c062540 con 0x7f65b409d6c0 2024-09-13T17:42:23.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.968+0000 7f65b94e8640 1 -- 172.21.15.102:0/3105481865 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65b4064000 con 0x7f65b409d6c0 2024-09-13T17:42:23.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:23.972+0000 7f65aaffd640 1 -- 172.21.15.102:0/3105481865 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f659c049080 con 0x7f65b409d6c0 2024-09-13T17:42:24.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.126+0000 7f65b94e8640 1 -- 172.21.15.102:0/3105481865 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f65b4099a80 con 0x7f65b409d6c0 2024-09-13T17:42:24.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.126+0000 7f65aaffd640 1 -- 172.21.15.102:0/3105481865 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f659c037070 con 0x7f65b409d6c0 2024-09-13T17:42:24.128 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-09-13T17:42:24.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.128+0000 7f65b94e8640 1 -- 172.21.15.102:0/3105481865 >> v1:172.21.15.102:6816/34937 conn(0x7f6594031160 legacy=0x7f6594033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:24.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.128+0000 7f65b94e8640 1 -- 172.21.15.102:0/3105481865 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65b409d6c0 msgr2=0x7f65b411c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:24.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.128+0000 7f65b94e8640 1 --2- 172.21.15.102:0/3105481865 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65b409d6c0 0x7f65b411c2d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f659c031000 tx=0x7f659c032000 comp rx=0 tx=0).stop 2024-09-13T17:42:24.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.128+0000 7f65b94e8640 1 -- 172.21.15.102:0/3105481865 shutdown_connections 2024-09-13T17:42:24.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.128+0000 7f65b94e8640 1 --2- 172.21.15.102:0/3105481865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65b412f3f0 0x7f65b411c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.128+0000 7f65b94e8640 1 --2- 172.21.15.102:0/3105481865 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65b409d6c0 0x7f65b411c2d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.128+0000 7f65b94e8640 1 --2- 172.21.15.102:0/3105481865 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65b4067130 0x7f65b411bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.128+0000 7f65b94e8640 1 -- 172.21.15.102:0/3105481865 >> 172.21.15.102:0/3105481865 conn(0x7f65b40a0420 msgr2=0x7f65b40988f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:24.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.128+0000 7f65b94e8640 1 -- 172.21.15.102:0/3105481865 shutdown_connections 2024-09-13T17:42:24.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.128+0000 7f65b94e8640 1 -- 172.21.15.102:0/3105481865 wait complete. 2024-09-13T17:42:24.141 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:42:24.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.233+0000 7fe112280640 1 Processor -- start 2024-09-13T17:42:24.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.233+0000 7fe112280640 1 -- start start 2024-09-13T17:42:24.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.234+0000 7fe112280640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe10c128330 0x7fe10c128700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:24.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.234+0000 7fe112280640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe10c128c40 0x7fe10c12b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:24.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.234+0000 7fe112280640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe10c0991b0 0x7fe10c12bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:24.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.234+0000 7fe112280640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe10c097be0 con 0x7fe10c0991b0 2024-09-13T17:42:24.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.234+0000 7fe112280640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe10c098e70 con 0x7fe10c128c40 2024-09-13T17:42:24.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.234+0000 7fe112280640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe10c098310 con 0x7fe10c128330 2024-09-13T17:42:24.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.234+0000 7fe11081d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe10c0991b0 0x7fe10c12bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:24.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.234+0000 7fe10bfff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe10c128330 0x7fe10c128700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:24.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.234+0000 7fe10b7fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe10c128c40 0x7fe10c12b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:24.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.234+0000 7fe11081d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe10c0991b0 0x7fe10c12bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58082/0 (socket says 172.21.15.102:58082) 2024-09-13T17:42:24.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.234+0000 7fe11081d640 1 -- 172.21.15.102:0/665384084 learned_addr learned my addr 172.21.15.102:0/665384084 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:24.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.234+0000 7fe10bfff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe10c128330 0x7fe10c128700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48906/0 (socket says 172.21.15.102:48906) 2024-09-13T17:42:24.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.234+0000 7fe11081d640 1 -- 172.21.15.102:0/665384084 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe10c128330 msgr2=0x7fe10c128700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:24.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.234+0000 7fe11081d640 1 --2- 172.21.15.102:0/665384084 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe10c128330 0x7fe10c128700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.234+0000 7fe11081d640 1 -- 172.21.15.102:0/665384084 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe10c128c40 msgr2=0x7fe10c12b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:24.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.234+0000 7fe11081d640 1 --2- 172.21.15.102:0/665384084 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe10c128c40 0x7fe10c12b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.234+0000 7fe11081d640 1 -- 172.21.15.102:0/665384084 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10c0985d0 con 0x7fe10c0991b0 2024-09-13T17:42:24.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.234+0000 7fe10bfff640 1 --2- 172.21.15.102:0/665384084 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe10c128330 0x7fe10c128700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:24.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.234+0000 7fe10b7fe640 1 --2- 172.21.15.102:0/665384084 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe10c128c40 0x7fe10c12b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:24.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.235+0000 7fe11081d640 1 --2- 172.21.15.102:0/665384084 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe10c0991b0 0x7fe10c12bd80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe104009f20 tx=0x7fe104009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe82e6dc8e1813e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:24.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.235+0000 7fe10affd640 1 -- 172.21.15.102:0/665384084 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe104031400 con 0x7fe10c0991b0 2024-09-13T17:42:24.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.235+0000 7fe10affd640 1 -- 172.21.15.102:0/665384084 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10402d050 con 0x7fe10c0991b0 2024-09-13T17:42:24.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.235+0000 7fe10affd640 1 -- 172.21.15.102:0/665384084 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1040317d0 con 0x7fe10c0991b0 2024-09-13T17:42:24.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.236+0000 7fe112280640 1 -- 172.21.15.102:0/665384084 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe10c0991b0 msgr2=0x7fe10c12bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:24.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.236+0000 7fe112280640 1 --2- 172.21.15.102:0/665384084 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe10c0991b0 0x7fe10c12bd80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe104009f20 tx=0x7fe104009840 comp rx=0 tx=0).stop 2024-09-13T17:42:24.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.236+0000 7fe112280640 1 -- 172.21.15.102:0/665384084 shutdown_connections 2024-09-13T17:42:24.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.236+0000 7fe112280640 1 --2- 172.21.15.102:0/665384084 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe10c0991b0 0x7fe10c12bd80 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.236+0000 7fe112280640 1 --2- 172.21.15.102:0/665384084 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe10c128c40 0x7fe10c12b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.236+0000 7fe112280640 1 --2- 172.21.15.102:0/665384084 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe10c128330 0x7fe10c128700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.236+0000 7fe112280640 1 -- 172.21.15.102:0/665384084 >> 172.21.15.102:0/665384084 conn(0x7fe10c09e820 msgr2=0x7fe10c09ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:24.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.236+0000 7fe112280640 1 -- 172.21.15.102:0/665384084 shutdown_connections 2024-09-13T17:42:24.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.236+0000 7fe112280640 1 -- 172.21.15.102:0/665384084 wait complete. 2024-09-13T17:42:24.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.237+0000 7fe112280640 1 Processor -- start 2024-09-13T17:42:24.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.237+0000 7fe112280640 1 -- start start 2024-09-13T17:42:24.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.237+0000 7fe112280640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe10c0991b0 0x7fe10c115bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:24.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.237+0000 7fe112280640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe10c128330 0x7fe10c06c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:24.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.237+0000 7fe112280640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe10c128c40 0x7fe10c06c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:24.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.237+0000 7fe112280640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10c09bd80 con 0x7fe10c128c40 2024-09-13T17:42:24.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.237+0000 7fe112280640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe10c06ec10 con 0x7fe10c128330 2024-09-13T17:42:24.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.237+0000 7fe112280640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe10c1153e0 con 0x7fe10c0991b0 2024-09-13T17:42:24.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.237+0000 7fe10bfff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe10c0991b0 0x7fe10c115bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:24.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.237+0000 7fe10b7fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe10c128330 0x7fe10c06c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:24.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.237+0000 7fe11081d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe10c128c40 0x7fe10c06c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:24.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.237+0000 7fe10bfff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe10c0991b0 0x7fe10c115bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48918/0 (socket says 172.21.15.102:48918) 2024-09-13T17:42:24.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.237+0000 7fe10bfff640 1 -- 172.21.15.102:0/754621171 learned_addr learned my addr 172.21.15.102:0/754621171 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:24.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.237+0000 7fe11081d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe10c128c40 0x7fe10c06c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58098/0 (socket says 172.21.15.102:58098) 2024-09-13T17:42:24.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.238+0000 7fe10b7fe640 1 -- 172.21.15.102:0/754621171 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe10c0991b0 msgr2=0x7fe10c115bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:24.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.238+0000 7fe10b7fe640 1 --2- 172.21.15.102:0/754621171 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe10c0991b0 0x7fe10c115bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.238+0000 7fe10b7fe640 1 -- 172.21.15.102:0/754621171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe10c128c40 msgr2=0x7fe10c06c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:24.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.238+0000 7fe10b7fe640 1 --2- 172.21.15.102:0/754621171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe10c128c40 0x7fe10c06c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.238+0000 7fe10b7fe640 1 -- 172.21.15.102:0/754621171 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0fc009390 con 0x7fe10c128330 2024-09-13T17:42:24.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.238+0000 7fe11081d640 1 --2- 172.21.15.102:0/754621171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe10c128c40 0x7fe10c06c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:24.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.238+0000 7fe10b7fe640 1 --2- 172.21.15.102:0/754621171 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe10c128330 0x7fe10c06c0d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe0f400ea20 tx=0x7fe0f40090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:24.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.238+0000 7fe103fff640 1 -- 172.21.15.102:0/754621171 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f400ca10 con 0x7fe10c128330 2024-09-13T17:42:24.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.238+0000 7fe103fff640 1 -- 172.21.15.102:0/754621171 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f400cb70 con 0x7fe10c128330 2024-09-13T17:42:24.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.238+0000 7fe112280640 1 -- 172.21.15.102:0/754621171 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0f4009390 con 0x7fe10c128330 2024-09-13T17:42:24.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.238+0000 7fe103fff640 1 -- 172.21.15.102:0/754621171 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f4013470 con 0x7fe10c128330 2024-09-13T17:42:24.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.238+0000 7fe112280640 1 -- 172.21.15.102:0/754621171 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe104009ae0 con 0x7fe10c128330 2024-09-13T17:42:24.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.239+0000 7fe103fff640 1 -- 172.21.15.102:0/754621171 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fe0f40137e0 con 0x7fe10c128330 2024-09-13T17:42:24.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.239+0000 7fe103fff640 1 -- 172.21.15.102:0/754621171 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3660+0+0 (secure 0 0 0) 0x7fe0f40423c0 con 0x7fe10c128330 2024-09-13T17:42:24.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.239+0000 7fe112280640 1 -- 172.21.15.102:0/754621171 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe10c115220 con 0x7fe10c128330 2024-09-13T17:42:24.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.243+0000 7fe103fff640 1 -- 172.21.15.102:0/754621171 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fe0f4012730 con 0x7fe10c128330 2024-09-13T17:42:24.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.409+0000 7fe112280640 1 -- 172.21.15.102:0/754621171 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe10c064000 con 0x7fe10c128330 2024-09-13T17:42:24.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.409+0000 7fe103fff640 1 -- 172.21.15.102:0/754621171 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe0f402a080 con 0x7fe10c128330 2024-09-13T17:42:24.411 INFO:teuthology.orchestra.run.smithi102.stderr:listed 0 entries 2024-09-13T17:42:24.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.411+0000 7fe112280640 1 -- 172.21.15.102:0/754621171 >> v1:172.21.15.102:6816/34937 conn(0x7fe0ec031160 legacy=0x7fe0ec033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:24.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.411+0000 7fe112280640 1 -- 172.21.15.102:0/754621171 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe10c128330 msgr2=0x7fe10c06c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:24.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.411+0000 7fe112280640 1 --2- 172.21.15.102:0/754621171 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe10c128330 0x7fe10c06c0d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe0f400ea20 tx=0x7fe0f40090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.411+0000 7fe112280640 1 -- 172.21.15.102:0/754621171 shutdown_connections 2024-09-13T17:42:24.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.411+0000 7fe112280640 1 --2- 172.21.15.102:0/754621171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe10c128c40 0x7fe10c06c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.411+0000 7fe112280640 1 --2- 172.21.15.102:0/754621171 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe10c128330 0x7fe10c06c0d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.411+0000 7fe112280640 1 --2- 172.21.15.102:0/754621171 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe10c0991b0 0x7fe10c115bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.411+0000 7fe112280640 1 -- 172.21.15.102:0/754621171 >> 172.21.15.102:0/754621171 conn(0x7fe10c09e820 msgr2=0x7fe10c126b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:24.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.411+0000 7fe112280640 1 -- 172.21.15.102:0/754621171 shutdown_connections 2024-09-13T17:42:24.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.411+0000 7fe112280640 1 -- 172.21.15.102:0/754621171 wait complete. 2024-09-13T17:42:24.424 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T17:42:24.424 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:42:24.444 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:24.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.535+0000 7fafcd75d640 1 Processor -- start 2024-09-13T17:42:24.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.535+0000 7fafcd75d640 1 -- start start 2024-09-13T17:42:24.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.536+0000 7fafcd75d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fafc80a1320 0x7fafc8134770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:24.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.536+0000 7fafcd75d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fafc80a1740 0x7fafc8136cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:24.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.536+0000 7fafcd75d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafc8069260 con 0x7fafc8068920 2024-09-13T17:42:24.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.536+0000 7fafcd75d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafc8068cf0 con 0x7fafc80a1320 2024-09-13T17:42:24.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.536+0000 7fafcd75d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fafc8069630 con 0x7fafc80a1740 2024-09-13T17:42:24.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.536+0000 7fafc67fc640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fafc80a1320 0x7fafc8134770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:24.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.536+0000 7fafc77fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fafc80a1740 0x7fafc8136cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:24.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.536+0000 7fafc77fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fafc80a1740 0x7fafc8136cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48932/0 (socket says 172.21.15.102:48932) 2024-09-13T17:42:24.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.536+0000 7fafc6ffd640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fafc8068920 0x7fafc80a0c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:47392/0 (socket says 172.21.15.102:47392) 2024-09-13T17:42:24.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.536+0000 7fafc67fc640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fafc80a1320 0x7fafc8134770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58102/0 (socket says 172.21.15.102:58102) 2024-09-13T17:42:24.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.536+0000 7fafc77fe640 1 -- 172.21.15.102:0/281853292 learned_addr learned my addr 172.21.15.102:0/281853292 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:24.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.536+0000 7fafc67fc640 1 -- 172.21.15.102:0/281853292 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fafc80a1740 msgr2=0x7fafc8136cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:24.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.536+0000 7fafc67fc640 1 --2- 172.21.15.102:0/281853292 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fafc80a1740 0x7fafc8136cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.536+0000 7fafc67fc640 1 -- 172.21.15.102:0/281853292 >> v1:172.21.15.102:6789/0 conn(0x7fafc8068920 legacy=0x7fafc80a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:24.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.536+0000 7fafc67fc640 1 -- 172.21.15.102:0/281853292 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc8069870 con 0x7fafc80a1320 2024-09-13T17:42:24.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.536+0000 7fafc5ffb640 1 -- 172.21.15.102:0/281853292 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1638051022 0 0) 0x7fafc8069260 con 0x7fafc8068920 2024-09-13T17:42:24.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.536+0000 7fafc77fe640 1 --2- 172.21.15.102:0/281853292 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fafc80a1740 0x7fafc8136cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:24.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.537+0000 7fafc67fc640 1 --2- 172.21.15.102:0/281853292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafc80a1320 0x7fafc8134770 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fafbc009850 tx=0x7fafbc0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea7172bfd29f077 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:24.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.537+0000 7fafc5ffb640 1 -- 172.21.15.102:0/281853292 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafbc02ac80 con 0x7fafc80a1320 2024-09-13T17:42:24.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.537+0000 7fafc5ffb640 1 -- 172.21.15.102:0/281853292 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafbc02ade0 con 0x7fafc80a1320 2024-09-13T17:42:24.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.537+0000 7fafc5ffb640 1 -- 172.21.15.102:0/281853292 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafbc032670 con 0x7fafc80a1320 2024-09-13T17:42:24.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.538+0000 7fafcd75d640 1 -- 172.21.15.102:0/281853292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafc80a1320 msgr2=0x7fafc8134770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:24.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.538+0000 7fafcd75d640 1 --2- 172.21.15.102:0/281853292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafc80a1320 0x7fafc8134770 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fafbc009850 tx=0x7fafbc0090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.538+0000 7fafcd75d640 1 -- 172.21.15.102:0/281853292 shutdown_connections 2024-09-13T17:42:24.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.538+0000 7fafcd75d640 1 --2- 172.21.15.102:0/281853292 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fafc80a1740 0x7fafc8136cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.538+0000 7fafcd75d640 1 --2- 172.21.15.102:0/281853292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafc80a1320 0x7fafc8134770 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.538+0000 7fafcd75d640 1 -- 172.21.15.102:0/281853292 >> 172.21.15.102:0/281853292 conn(0x7fafc80a34c0 msgr2=0x7fafc806e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:24.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.538+0000 7fafcd75d640 1 -- 172.21.15.102:0/281853292 shutdown_connections 2024-09-13T17:42:24.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.538+0000 7fafcd75d640 1 -- 172.21.15.102:0/281853292 wait complete. 2024-09-13T17:42:24.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.539+0000 7fafcd75d640 1 Processor -- start 2024-09-13T17:42:24.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.539+0000 7fafcd75d640 1 -- start start 2024-09-13T17:42:24.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.539+0000 7fafcd75d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fafc8068920 0x7fafc812fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:24.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.539+0000 7fafcd75d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafc80a1320 0x7fafc8119060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:24.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.539+0000 7fafcd75d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fafc8135280 0x7fafc81195a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:24.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.539+0000 7fafcd75d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc811b6a0 con 0x7fafc80a1320 2024-09-13T17:42:24.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.539+0000 7fafcd75d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc80a1d50 con 0x7fafc8135280 2024-09-13T17:42:24.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.539+0000 7fafcd75d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fafc8136270 con 0x7fafc8068920 2024-09-13T17:42:24.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.539+0000 7fafc6ffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fafc8068920 0x7fafc812fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:24.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.539+0000 7fafc67fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafc80a1320 0x7fafc8119060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:24.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.539+0000 7fafc67fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafc80a1320 0x7fafc8119060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58108/0 (socket says 172.21.15.102:58108) 2024-09-13T17:42:24.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.539+0000 7fafc6ffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fafc8068920 0x7fafc812fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48940/0 (socket says 172.21.15.102:48940) 2024-09-13T17:42:24.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.539+0000 7fafc67fc640 1 -- 172.21.15.102:0/2580412534 learned_addr learned my addr 172.21.15.102:0/2580412534 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:24.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.539+0000 7fafc77fe640 1 --2- 172.21.15.102:0/2580412534 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fafc8135280 0x7fafc81195a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:24.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.540+0000 7fafc67fc640 1 -- 172.21.15.102:0/2580412534 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fafc8068920 msgr2=0x7fafc812fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:24.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.540+0000 7fafc67fc640 1 --2- 172.21.15.102:0/2580412534 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fafc8068920 0x7fafc812fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.540+0000 7fafc67fc640 1 -- 172.21.15.102:0/2580412534 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fafc8135280 msgr2=0x7fafc81195a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:24.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.540+0000 7fafc67fc640 1 --2- 172.21.15.102:0/2580412534 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fafc8135280 0x7fafc81195a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.540+0000 7fafc67fc640 1 -- 172.21.15.102:0/2580412534 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc0009ae0 con 0x7fafc80a1320 2024-09-13T17:42:24.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.540+0000 7fafc6ffd640 1 --2- 172.21.15.102:0/2580412534 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fafc8068920 0x7fafc812fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:24.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.540+0000 7fafc77fe640 1 --2- 172.21.15.102:0/2580412534 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fafc8135280 0x7fafc81195a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:24.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.540+0000 7fafc67fc640 1 --2- 172.21.15.102:0/2580412534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafc80a1320 0x7fafc8119060 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fafbc009850 tx=0x7fafbc032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:24.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.540+0000 7fafaeffd640 1 -- 172.21.15.102:0/2580412534 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafbc03e660 con 0x7fafc80a1320 2024-09-13T17:42:24.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.540+0000 7fafaeffd640 1 -- 172.21.15.102:0/2580412534 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafbc03e7c0 con 0x7fafc80a1320 2024-09-13T17:42:24.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.540+0000 7fafcd75d640 1 -- 172.21.15.102:0/2580412534 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafbc009390 con 0x7fafc80a1320 2024-09-13T17:42:24.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.540+0000 7fafaeffd640 1 -- 172.21.15.102:0/2580412534 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafbc03ebc0 con 0x7fafc80a1320 2024-09-13T17:42:24.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.540+0000 7fafcd75d640 1 -- 172.21.15.102:0/2580412534 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc8130180 con 0x7fafc80a1320 2024-09-13T17:42:24.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.541+0000 7fafaeffd640 1 -- 172.21.15.102:0/2580412534 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fafbc03c400 con 0x7fafc80a1320 2024-09-13T17:42:24.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.541+0000 7fafaeffd640 1 -- 172.21.15.102:0/2580412534 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3660+0+0 (secure 0 0 0) 0x7fafbc0620e0 con 0x7fafc80a1320 2024-09-13T17:42:24.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.541+0000 7fafcd75d640 1 -- 172.21.15.102:0/2580412534 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafc8064000 con 0x7fafc80a1320 2024-09-13T17:42:24.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.545+0000 7fafaeffd640 1 -- 172.21.15.102:0/2580412534 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fafbc037080 con 0x7fafc80a1320 2024-09-13T17:42:24.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.699+0000 7fafcd75d640 1 -- 172.21.15.102:0/2580412534 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fafc8131c10 con 0x7fafc80a1320 2024-09-13T17:42:24.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.700+0000 7fafaeffd640 1 -- 172.21.15.102:0/2580412534 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+11330 (secure 0 0 0) 0x7fafbc043020 con 0x7fafc80a1320 2024-09-13T17:42:24.702 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:42:24.702 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":21,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:42:20.337430+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:04.672265+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:05.583134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:24.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.702+0000 7fafcd75d640 1 -- 172.21.15.102:0/2580412534 >> v1:172.21.15.102:6816/34937 conn(0x7fafb4031160 legacy=0x7fafb4033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:24.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.702+0000 7fafcd75d640 1 -- 172.21.15.102:0/2580412534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafc80a1320 msgr2=0x7fafc8119060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:24.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.702+0000 7fafcd75d640 1 --2- 172.21.15.102:0/2580412534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafc80a1320 0x7fafc8119060 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fafbc009850 tx=0x7fafbc032000 comp rx=0 tx=0).stop 2024-09-13T17:42:24.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.702+0000 7fafcd75d640 1 -- 172.21.15.102:0/2580412534 shutdown_connections 2024-09-13T17:42:24.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.702+0000 7fafcd75d640 1 --2- 172.21.15.102:0/2580412534 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fafc8135280 0x7fafc81195a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.702+0000 7fafcd75d640 1 --2- 172.21.15.102:0/2580412534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafc80a1320 0x7fafc8119060 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.702+0000 7fafcd75d640 1 --2- 172.21.15.102:0/2580412534 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fafc8068920 0x7fafc812fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.702+0000 7fafcd75d640 1 -- 172.21.15.102:0/2580412534 >> 172.21.15.102:0/2580412534 conn(0x7fafc80a34c0 msgr2=0x7fafc809c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:24.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.702+0000 7fafcd75d640 1 -- 172.21.15.102:0/2580412534 shutdown_connections 2024-09-13T17:42:24.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.702+0000 7fafcd75d640 1 -- 172.21.15.102:0/2580412534 wait complete. 2024-09-13T17:42:24.714 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:54.442612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T17:41:59.076782+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 16, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:42:04.672265+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:42:05.583134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:42:24.715 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:42:24.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.810+0000 7f25d6e90640 1 Processor -- start 2024-09-13T17:42:24.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.810+0000 7f25d6e90640 1 -- start start 2024-09-13T17:42:24.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.810+0000 7f25d6e90640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f25d0098ed0 0x7f25d0131610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:24.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.810+0000 7f25d6e90640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f25d0131b50 0x7f25d0133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:24.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.810+0000 7f25d6e90640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25d00999e0 con 0x7f25d0096a70 2024-09-13T17:42:24.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.810+0000 7f25d6e90640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d0069c30 con 0x7f25d0131b50 2024-09-13T17:42:24.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.810+0000 7f25d6e90640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f25d006ed00 con 0x7f25d0098ed0 2024-09-13T17:42:24.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.810+0000 7f25d4c2c640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f25d0096a70 0x7f25d00987c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:47396/0 (socket says 172.21.15.102:47396) 2024-09-13T17:42:24.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.810+0000 7f25d4c2c640 1 -- 172.21.15.102:0/645226854 learned_addr learned my addr 172.21.15.102:0/645226854 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:42:24.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.810+0000 7f25cffff640 1 --2- 172.21.15.102:0/645226854 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f25d0098ed0 0x7f25d0131610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:24.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.810+0000 7f25d542d640 1 --2- 172.21.15.102:0/645226854 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f25d0131b50 0x7f25d0133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:24.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.811+0000 7f25cffff640 1 -- 172.21.15.102:0/645226854 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f25d0131b50 msgr2=0x7f25d0133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:24.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.811+0000 7f25cf7fe640 1 -- 172.21.15.102:0/645226854 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2161566913 0 0) 0x7f25d00999e0 con 0x7f25d0096a70 2024-09-13T17:42:24.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.811+0000 7f25cffff640 1 --2- 172.21.15.102:0/645226854 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f25d0131b50 0x7f25d0133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.811+0000 7f25cffff640 1 -- 172.21.15.102:0/645226854 >> v1:172.21.15.102:6789/0 conn(0x7f25d0096a70 legacy=0x7f25d00987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:24.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.811+0000 7f25cffff640 1 -- 172.21.15.102:0/645226854 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d00997a0 con 0x7f25d0098ed0 2024-09-13T17:42:24.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.811+0000 7f25d542d640 1 --2- 172.21.15.102:0/645226854 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f25d0131b50 0x7f25d0133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:24.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.811+0000 7f25cffff640 1 --2- 172.21.15.102:0/645226854 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f25d0098ed0 0x7f25d0131610 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f25bc009880 tx=0x7f25bc0090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ac7e5f1a873bb7d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:24.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.812+0000 7f25cf7fe640 1 -- 172.21.15.102:0/645226854 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc034440 con 0x7f25d0098ed0 2024-09-13T17:42:24.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.812+0000 7f25cf7fe640 1 -- 172.21.15.102:0/645226854 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25bc0345a0 con 0x7f25d0098ed0 2024-09-13T17:42:24.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.812+0000 7f25cf7fe640 1 -- 172.21.15.102:0/645226854 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc034940 con 0x7f25d0098ed0 2024-09-13T17:42:24.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.812+0000 7f25d6e90640 1 -- 172.21.15.102:0/645226854 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f25d0098ed0 msgr2=0x7f25d0131610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:24.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.812+0000 7f25d6e90640 1 --2- 172.21.15.102:0/645226854 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f25d0098ed0 0x7f25d0131610 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f25bc009880 tx=0x7f25bc0090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.812+0000 7f25d6e90640 1 -- 172.21.15.102:0/645226854 shutdown_connections 2024-09-13T17:42:24.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.812+0000 7f25d6e90640 1 --2- 172.21.15.102:0/645226854 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f25d0131b50 0x7f25d0133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.812+0000 7f25d6e90640 1 --2- 172.21.15.102:0/645226854 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f25d0098ed0 0x7f25d0131610 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.812+0000 7f25d6e90640 1 -- 172.21.15.102:0/645226854 >> 172.21.15.102:0/645226854 conn(0x7f25d009e720 msgr2=0x7f25d009eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:24.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.812+0000 7f25d6e90640 1 -- 172.21.15.102:0/645226854 shutdown_connections 2024-09-13T17:42:24.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.812+0000 7f25d6e90640 1 -- 172.21.15.102:0/645226854 wait complete. 2024-09-13T17:42:24.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.813+0000 7f25d6e90640 1 Processor -- start 2024-09-13T17:42:24.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.813+0000 7f25d6e90640 1 -- start start 2024-09-13T17:42:24.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.813+0000 7f25d6e90640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25d0096a70 0x7f25d011ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:24.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.813+0000 7f25d6e90640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f25d0098ed0 0x7f25d006c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:24.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.813+0000 7f25d6e90640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f25d0131b50 0x7f25d006c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:24.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.813+0000 7f25d6e90640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d006e6a0 con 0x7f25d0096a70 2024-09-13T17:42:24.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.813+0000 7f25d6e90640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d0119e50 con 0x7f25d0098ed0 2024-09-13T17:42:24.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.813+0000 7f25d6e90640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f25d011a260 con 0x7f25d0131b50 2024-09-13T17:42:24.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.814+0000 7f25d4c2c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25d0096a70 0x7f25d011ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:24.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.814+0000 7f25cffff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f25d0098ed0 0x7f25d006c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:24.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.814+0000 7f25d542d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f25d0131b50 0x7f25d006c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:24.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.814+0000 7f25d4c2c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25d0096a70 0x7f25d011ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58112/0 (socket says 172.21.15.102:58112) 2024-09-13T17:42:24.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.814+0000 7f25d4c2c640 1 -- 172.21.15.102:0/750555082 learned_addr learned my addr 172.21.15.102:0/750555082 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:24.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.814+0000 7f25cffff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f25d0098ed0 0x7f25d006c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:39688/0 (socket says 172.21.15.102:39688) 2024-09-13T17:42:24.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.814+0000 7f25d542d640 1 -- 172.21.15.102:0/750555082 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f25d0098ed0 msgr2=0x7f25d006c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:24.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.814+0000 7f25d542d640 1 --2- 172.21.15.102:0/750555082 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f25d0098ed0 0x7f25d006c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.814+0000 7f25d542d640 1 -- 172.21.15.102:0/750555082 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25d0096a70 msgr2=0x7f25d011ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:24.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.814+0000 7f25d542d640 1 --2- 172.21.15.102:0/750555082 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25d0096a70 0x7f25d011ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.814+0000 7f25d542d640 1 -- 172.21.15.102:0/750555082 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c8009a20 con 0x7f25d0131b50 2024-09-13T17:42:24.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.814+0000 7f25d4c2c640 1 --2- 172.21.15.102:0/750555082 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25d0096a70 0x7f25d011ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:24.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.814+0000 7f25cffff640 1 --2- 172.21.15.102:0/750555082 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f25d0098ed0 0x7f25d006c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:24.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.814+0000 7f25d542d640 1 --2- 172.21.15.102:0/750555082 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f25d0131b50 0x7f25d006c850 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f25d0069420 tx=0x7f25c800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:24.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.814+0000 7f25ccff9640 1 -- 172.21.15.102:0/750555082 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c800ce10 con 0x7f25d0131b50 2024-09-13T17:42:24.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.814+0000 7f25ccff9640 1 -- 172.21.15.102:0/750555082 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c800f040 con 0x7f25d0131b50 2024-09-13T17:42:24.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.814+0000 7f25ccff9640 1 -- 172.21.15.102:0/750555082 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c80146d0 con 0x7f25d0131b50 2024-09-13T17:42:24.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.814+0000 7f25d6e90640 1 -- 172.21.15.102:0/750555082 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25bc009390 con 0x7f25d0131b50 2024-09-13T17:42:24.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.814+0000 7f25d6e90640 1 -- 172.21.15.102:0/750555082 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d009a490 con 0x7f25d0131b50 2024-09-13T17:42:24.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.815+0000 7f25ccff9640 1 -- 172.21.15.102:0/750555082 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f25c8003020 con 0x7f25d0131b50 2024-09-13T17:42:24.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.815+0000 7f25ccff9640 1 -- 172.21.15.102:0/750555082 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3660+0+0 (secure 0 0 0) 0x7f25c80429f0 con 0x7f25d0131b50 2024-09-13T17:42:24.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.815+0000 7f25d6e90640 1 -- 172.21.15.102:0/750555082 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25d006df60 con 0x7f25d0131b50 2024-09-13T17:42:24.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.819+0000 7f25ccff9640 1 -- 172.21.15.102:0/750555082 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f25c8018080 con 0x7f25d0131b50 2024-09-13T17:42:24.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.969+0000 7f25d6e90640 1 -- 172.21.15.102:0/750555082 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f25d00676b0 con 0x7f25d0131b50 2024-09-13T17:42:24.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.969+0000 7f25ccff9640 1 -- 172.21.15.102:0/750555082 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f25c80112c0 con 0x7f25d0131b50 2024-09-13T17:42:24.971 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-09-13T17:42:24.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.971+0000 7f25d6e90640 1 -- 172.21.15.102:0/750555082 >> v1:172.21.15.102:6816/34937 conn(0x7f25b4031160 legacy=0x7f25b4033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:24.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.971+0000 7f25d6e90640 1 -- 172.21.15.102:0/750555082 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f25d0131b50 msgr2=0x7f25d006c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:24.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.971+0000 7f25d6e90640 1 --2- 172.21.15.102:0/750555082 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f25d0131b50 0x7f25d006c850 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f25d0069420 tx=0x7f25c800b040 comp rx=0 tx=0).stop 2024-09-13T17:42:24.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.971+0000 7f25d6e90640 1 -- 172.21.15.102:0/750555082 shutdown_connections 2024-09-13T17:42:24.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.971+0000 7f25d6e90640 1 --2- 172.21.15.102:0/750555082 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f25d0131b50 0x7f25d006c850 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.971+0000 7f25d6e90640 1 --2- 172.21.15.102:0/750555082 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f25d0098ed0 0x7f25d006c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.971+0000 7f25d6e90640 1 --2- 172.21.15.102:0/750555082 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25d0096a70 0x7f25d011ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.971+0000 7f25d6e90640 1 -- 172.21.15.102:0/750555082 >> 172.21.15.102:0/750555082 conn(0x7f25d009e720 msgr2=0x7f25d00971f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:24.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.971+0000 7f25d6e90640 1 -- 172.21.15.102:0/750555082 shutdown_connections 2024-09-13T17:42:24.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:24.971+0000 7f25d6e90640 1 -- 172.21.15.102:0/750555082 wait complete. 2024-09-13T17:42:24.984 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-13T17:42:25.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.077+0000 7f01ae31d640 1 Processor -- start 2024-09-13T17:42:25.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.077+0000 7f01ae31d640 1 -- start start 2024-09-13T17:42:25.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.078+0000 7f01ae31d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f01a8096a70 0x7f01a80987c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.078+0000 7f01ae31d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f01a8098d00 0x7f01a812c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.078+0000 7f01ae31d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f01a8099110 0x7f01a812c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.078+0000 7f01ae31d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01a8069c30 con 0x7f01a8098d00 2024-09-13T17:42:25.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.078+0000 7f01ae31d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a806ed00 con 0x7f01a8099110 2024-09-13T17:42:25.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.078+0000 7f01ae31d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f01a80999e0 con 0x7f01a8096a70 2024-09-13T17:42:25.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.078+0000 7f01a7fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f01a8096a70 0x7f01a80987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.078+0000 7f01a77fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f01a8098d00 0x7f01a812c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.078+0000 7f01ac8ba640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f01a8099110 0x7f01a812c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.078+0000 7f01a7fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f01a8096a70 0x7f01a80987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48966/0 (socket says 172.21.15.102:48966) 2024-09-13T17:42:25.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.078+0000 7f01a7fff640 1 -- 172.21.15.102:0/1509290696 learned_addr learned my addr 172.21.15.102:0/1509290696 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:25.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.078+0000 7f01a77fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f01a8098d00 0x7f01a812c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58114/0 (socket says 172.21.15.102:58114) 2024-09-13T17:42:25.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.078+0000 7f01a77fe640 1 -- 172.21.15.102:0/1509290696 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f01a8096a70 msgr2=0x7f01a80987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:25.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.078+0000 7f01a77fe640 1 --2- 172.21.15.102:0/1509290696 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f01a8096a70 0x7f01a80987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.078+0000 7f01a77fe640 1 -- 172.21.15.102:0/1509290696 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f01a8099110 msgr2=0x7f01a812c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:25.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.078+0000 7f01a77fe640 1 --2- 172.21.15.102:0/1509290696 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f01a8099110 0x7f01a812c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.078+0000 7f01a77fe640 1 -- 172.21.15.102:0/1509290696 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a80997a0 con 0x7f01a8098d00 2024-09-13T17:42:25.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.078+0000 7f01a7fff640 1 --2- 172.21.15.102:0/1509290696 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f01a8096a70 0x7f01a80987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:25.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.078+0000 7f01ac8ba640 1 --2- 172.21.15.102:0/1509290696 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f01a8099110 0x7f01a812c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:25.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.079+0000 7f01a77fe640 1 --2- 172.21.15.102:0/1509290696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01a8098d00 0x7f01a812c2e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f019c009880 tx=0x7f019c0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64aeec129d5f645e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:25.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.079+0000 7f01a6ffd640 1 -- 172.21.15.102:0/1509290696 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c02ac40 con 0x7f01a8098d00 2024-09-13T17:42:25.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.079+0000 7f01a6ffd640 1 -- 172.21.15.102:0/1509290696 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019c02ada0 con 0x7f01a8098d00 2024-09-13T17:42:25.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.079+0000 7f01a6ffd640 1 -- 172.21.15.102:0/1509290696 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c032670 con 0x7f01a8098d00 2024-09-13T17:42:25.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.080+0000 7f01ae31d640 1 -- 172.21.15.102:0/1509290696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01a8098d00 msgr2=0x7f01a812c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:25.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.080+0000 7f01ae31d640 1 --2- 172.21.15.102:0/1509290696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01a8098d00 0x7f01a812c2e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f019c009880 tx=0x7f019c0090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.080+0000 7f01ae31d640 1 -- 172.21.15.102:0/1509290696 shutdown_connections 2024-09-13T17:42:25.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.080+0000 7f01ae31d640 1 --2- 172.21.15.102:0/1509290696 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f01a8099110 0x7f01a812c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.080+0000 7f01ae31d640 1 --2- 172.21.15.102:0/1509290696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01a8098d00 0x7f01a812c2e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.080+0000 7f01ae31d640 1 --2- 172.21.15.102:0/1509290696 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f01a8096a70 0x7f01a80987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.080+0000 7f01ae31d640 1 -- 172.21.15.102:0/1509290696 >> 172.21.15.102:0/1509290696 conn(0x7f01a809e720 msgr2=0x7f01a809eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:25.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.080+0000 7f01ae31d640 1 -- 172.21.15.102:0/1509290696 shutdown_connections 2024-09-13T17:42:25.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.080+0000 7f01ae31d640 1 -- 172.21.15.102:0/1509290696 wait complete. 2024-09-13T17:42:25.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.081+0000 7f01ae31d640 1 Processor -- start 2024-09-13T17:42:25.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.081+0000 7f01ae31d640 1 -- start start 2024-09-13T17:42:25.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.081+0000 7f01ae31d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f01a8096a70 0x7f01a811f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.081+0000 7f01ae31d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01a8098d00 0x7f01a811b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.081+0000 7f01ae31d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f01a8099110 0x7f01a811bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.081+0000 7f01ae31d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a811d820 con 0x7f01a8098d00 2024-09-13T17:42:25.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.081+0000 7f01ae31d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a81296f0 con 0x7f01a8096a70 2024-09-13T17:42:25.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.081+0000 7f01ae31d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f01a811ecc0 con 0x7f01a8099110 2024-09-13T17:42:25.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.081+0000 7f01a7fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f01a8096a70 0x7f01a811f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.081+0000 7f01a77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01a8098d00 0x7f01a811b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.081+0000 7f01a77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01a8098d00 0x7f01a811b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58126/0 (socket says 172.21.15.102:58126) 2024-09-13T17:42:25.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.081+0000 7f01a77fe640 1 -- 172.21.15.102:0/173930555 learned_addr learned my addr 172.21.15.102:0/173930555 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:25.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.081+0000 7f01ac8ba640 1 --2- 172.21.15.102:0/173930555 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f01a8099110 0x7f01a811bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.082+0000 7f01a77fe640 1 -- 172.21.15.102:0/173930555 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f01a8099110 msgr2=0x7f01a811bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:25.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.082+0000 7f01a77fe640 1 --2- 172.21.15.102:0/173930555 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f01a8099110 0x7f01a811bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.082+0000 7f01a77fe640 1 -- 172.21.15.102:0/173930555 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f01a8096a70 msgr2=0x7f01a811f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:25.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.082+0000 7f01a77fe640 1 --2- 172.21.15.102:0/173930555 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f01a8096a70 0x7f01a811f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.082+0000 7f01a77fe640 1 -- 172.21.15.102:0/173930555 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0198009450 con 0x7f01a8098d00 2024-09-13T17:42:25.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.082+0000 7f01ac8ba640 1 --2- 172.21.15.102:0/173930555 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f01a8099110 0x7f01a811bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:25.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.082+0000 7f01a77fe640 1 --2- 172.21.15.102:0/173930555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01a8098d00 0x7f01a811b870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f019c02a5b0 tx=0x7f019c032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:25.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.082+0000 7f018ffff640 1 -- 172.21.15.102:0/173930555 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c03d700 con 0x7f01a8098d00 2024-09-13T17:42:25.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.082+0000 7f018ffff640 1 -- 172.21.15.102:0/173930555 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019c03d860 con 0x7f01a8098d00 2024-09-13T17:42:25.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.082+0000 7f018ffff640 1 -- 172.21.15.102:0/173930555 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c03dc00 con 0x7f01a8098d00 2024-09-13T17:42:25.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.082+0000 7f01ae31d640 1 -- 172.21.15.102:0/173930555 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a000bc30 con 0x7f01a8098d00 2024-09-13T17:42:25.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.082+0000 7f01ae31d640 1 -- 172.21.15.102:0/173930555 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f019c009390 con 0x7f01a8098d00 2024-09-13T17:42:25.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.083+0000 7f018ffff640 1 -- 172.21.15.102:0/173930555 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f019c039040 con 0x7f01a8098d00 2024-09-13T17:42:25.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.083+0000 7f018ffff640 1 -- 172.21.15.102:0/173930555 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3660+0+0 (secure 0 0 0) 0x7f019c061a00 con 0x7f01a8098d00 2024-09-13T17:42:25.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.083+0000 7f01ae31d640 1 -- 172.21.15.102:0/173930555 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01a811ee00 con 0x7f01a8098d00 2024-09-13T17:42:25.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.087+0000 7f018ffff640 1 -- 172.21.15.102:0/173930555 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f019c042020 con 0x7f01a8098d00 2024-09-13T17:42:25.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.239+0000 7f01ae31d640 1 -- 172.21.15.102:0/173930555 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f01a8064000 con 0x7f01a8098d00 2024-09-13T17:42:25.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.240+0000 7f018ffff640 1 -- 172.21.15.102:0/173930555 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f019c037070 con 0x7f01a8098d00 2024-09-13T17:42:25.242 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-09-13T17:42:25.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.241+0000 7f01ae31d640 1 -- 172.21.15.102:0/173930555 >> v1:172.21.15.102:6816/34937 conn(0x7f0194031160 legacy=0x7f0194033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:25.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.241+0000 7f01ae31d640 1 -- 172.21.15.102:0/173930555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01a8098d00 msgr2=0x7f01a811b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:25.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.241+0000 7f01ae31d640 1 --2- 172.21.15.102:0/173930555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01a8098d00 0x7f01a811b870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f019c02a5b0 tx=0x7f019c032000 comp rx=0 tx=0).stop 2024-09-13T17:42:25.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.241+0000 7f01ae31d640 1 -- 172.21.15.102:0/173930555 shutdown_connections 2024-09-13T17:42:25.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.241+0000 7f01ae31d640 1 --2- 172.21.15.102:0/173930555 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f01a8099110 0x7f01a811bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.241+0000 7f01ae31d640 1 --2- 172.21.15.102:0/173930555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01a8098d00 0x7f01a811b870 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.241+0000 7f01ae31d640 1 --2- 172.21.15.102:0/173930555 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f01a8096a70 0x7f01a811f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.241+0000 7f01ae31d640 1 -- 172.21.15.102:0/173930555 >> 172.21.15.102:0/173930555 conn(0x7f01a809e720 msgr2=0x7f01a80971f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:25.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.241+0000 7f01ae31d640 1 -- 172.21.15.102:0/173930555 shutdown_connections 2024-09-13T17:42:25.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.241+0000 7f01ae31d640 1 -- 172.21.15.102:0/173930555 wait complete. 2024-09-13T17:42:25.254 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:42:25.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.348+0000 7f1f4a75d640 1 Processor -- start 2024-09-13T17:42:25.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.348+0000 7f1f4a75d640 1 -- start start 2024-09-13T17:42:25.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.349+0000 7f1f4a75d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1f44096a70 0x7f1f4412c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.349+0000 7f1f4a75d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1f4412c900 0x7f1f44098410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.349+0000 7f1f4a75d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1f4412ccd0 0x7f1f44098950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.349+0000 7f1f4a75d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f44096e40 con 0x7f1f44096a70 2024-09-13T17:42:25.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.349+0000 7f1f4a75d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f44099540 con 0x7f1f4412c900 2024-09-13T17:42:25.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.349+0000 7f1f4a75d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4409a8e0 con 0x7f1f4412ccd0 2024-09-13T17:42:25.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.349+0000 7f1f437fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1f4412c900 0x7f1f44098410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.349+0000 7f1f43fff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1f44096a70 0x7f1f4412c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.349+0000 7f1f48cfa640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1f4412ccd0 0x7f1f44098950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.349+0000 7f1f43fff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1f44096a70 0x7f1f4412c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58138/0 (socket says 172.21.15.102:58138) 2024-09-13T17:42:25.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.349+0000 7f1f437fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1f4412c900 0x7f1f44098410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:39720/0 (socket says 172.21.15.102:39720) 2024-09-13T17:42:25.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.349+0000 7f1f43fff640 1 -- 172.21.15.102:0/3555765115 learned_addr learned my addr 172.21.15.102:0/3555765115 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:25.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.349+0000 7f1f48cfa640 1 -- 172.21.15.102:0/3555765115 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1f4412c900 msgr2=0x7f1f44098410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:25.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.349+0000 7f1f48cfa640 1 --2- 172.21.15.102:0/3555765115 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1f4412c900 0x7f1f44098410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.349+0000 7f1f48cfa640 1 -- 172.21.15.102:0/3555765115 >> v2:172.21.15.102:3300/0 conn(0x7f1f44096a70 msgr2=0x7f1f4412c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:25.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.349+0000 7f1f48cfa640 1 --2- 172.21.15.102:0/3555765115 >> v2:172.21.15.102:3300/0 conn(0x7f1f44096a70 0x7f1f4412c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.349+0000 7f1f48cfa640 1 -- 172.21.15.102:0/3555765115 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f44099760 con 0x7f1f4412ccd0 2024-09-13T17:42:25.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.349+0000 7f1f43fff640 1 --2- 172.21.15.102:0/3555765115 >> v2:172.21.15.102:3300/0 conn(0x7f1f44096a70 0x7f1f4412c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:25.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.349+0000 7f1f437fe640 1 --2- 172.21.15.102:0/3555765115 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1f4412c900 0x7f1f44098410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:25.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.350+0000 7f1f48cfa640 1 --2- 172.21.15.102:0/3555765115 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1f4412ccd0 0x7f1f44098950 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009e90 tx=0x7f1f3c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47baa1eebcd12a9d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:25.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.350+0000 7f1f42ffd640 1 -- 172.21.15.102:0/3555765115 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c031400 con 0x7f1f4412ccd0 2024-09-13T17:42:25.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.350+0000 7f1f42ffd640 1 -- 172.21.15.102:0/3555765115 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3c02d050 con 0x7f1f4412ccd0 2024-09-13T17:42:25.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.350+0000 7f1f42ffd640 1 -- 172.21.15.102:0/3555765115 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c0317d0 con 0x7f1f4412ccd0 2024-09-13T17:42:25.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.351+0000 7f1f4a75d640 1 -- 172.21.15.102:0/3555765115 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1f4412ccd0 msgr2=0x7f1f44098950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:25.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.351+0000 7f1f4a75d640 1 --2- 172.21.15.102:0/3555765115 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1f4412ccd0 0x7f1f44098950 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009e90 tx=0x7f1f3c009780 comp rx=0 tx=0).stop 2024-09-13T17:42:25.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.351+0000 7f1f4a75d640 1 -- 172.21.15.102:0/3555765115 shutdown_connections 2024-09-13T17:42:25.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.351+0000 7f1f4a75d640 1 --2- 172.21.15.102:0/3555765115 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1f4412ccd0 0x7f1f44098950 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.351+0000 7f1f4a75d640 1 --2- 172.21.15.102:0/3555765115 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1f4412c900 0x7f1f44098410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.351+0000 7f1f4a75d640 1 --2- 172.21.15.102:0/3555765115 >> v2:172.21.15.102:3300/0 conn(0x7f1f44096a70 0x7f1f4412c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.351+0000 7f1f4a75d640 1 -- 172.21.15.102:0/3555765115 >> 172.21.15.102:0/3555765115 conn(0x7f1f4409e6b0 msgr2=0x7f1f4409eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:25.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.351+0000 7f1f4a75d640 1 -- 172.21.15.102:0/3555765115 shutdown_connections 2024-09-13T17:42:25.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.351+0000 7f1f4a75d640 1 -- 172.21.15.102:0/3555765115 wait complete. 2024-09-13T17:42:25.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.352+0000 7f1f4a75d640 1 Processor -- start 2024-09-13T17:42:25.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.352+0000 7f1f4a75d640 1 -- start start 2024-09-13T17:42:25.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.353+0000 7f1f4a75d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1f44096a70 0x7f1f4411a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.353+0000 7f1f4a75d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1f4412c900 0x7f1f4411ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.353+0000 7f1f4a75d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f4412ccd0 0x7f1f4413c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.353+0000 7f1f4a75d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f44097f30 con 0x7f1f4412ccd0 2024-09-13T17:42:25.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.353+0000 7f1f4a75d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4411c840 con 0x7f1f4412c900 2024-09-13T17:42:25.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.353+0000 7f1f4a75d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4411cb00 con 0x7f1f44096a70 2024-09-13T17:42:25.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.353+0000 7f1f437fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1f4412c900 0x7f1f4411ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.353+0000 7f1f43fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1f44096a70 0x7f1f4411a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.353+0000 7f1f437fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1f4412c900 0x7f1f4411ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:39728/0 (socket says 172.21.15.102:39728) 2024-09-13T17:42:25.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.353+0000 7f1f437fe640 1 -- 172.21.15.102:0/3894206384 learned_addr learned my addr 172.21.15.102:0/3894206384 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:25.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.353+0000 7f1f43fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1f44096a70 0x7f1f4411a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:48990/0 (socket says 172.21.15.102:48990) 2024-09-13T17:42:25.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.353+0000 7f1f48cfa640 1 --2- 172.21.15.102:0/3894206384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f4412ccd0 0x7f1f4413c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.353+0000 7f1f437fe640 1 -- 172.21.15.102:0/3894206384 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1f44096a70 msgr2=0x7f1f4411a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:25.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.353+0000 7f1f437fe640 1 --2- 172.21.15.102:0/3894206384 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1f44096a70 0x7f1f4411a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.353+0000 7f1f437fe640 1 -- 172.21.15.102:0/3894206384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f4412ccd0 msgr2=0x7f1f4413c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:25.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.353+0000 7f1f437fe640 1 --2- 172.21.15.102:0/3894206384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f4412ccd0 0x7f1f4413c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.353+0000 7f1f437fe640 1 -- 172.21.15.102:0/3894206384 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f34009450 con 0x7f1f4412c900 2024-09-13T17:42:25.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.353+0000 7f1f43fff640 1 --2- 172.21.15.102:0/3894206384 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1f44096a70 0x7f1f4411a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:25.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.353+0000 7f1f437fe640 1 --2- 172.21.15.102:0/3894206384 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1f4412c900 0x7f1f4411ad60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1f3800ec80 tx=0x7f1f380090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:25.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.353+0000 7f1f2bfff640 1 -- 172.21.15.102:0/3894206384 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3800eec0 con 0x7f1f4412c900 2024-09-13T17:42:25.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.353+0000 7f1f2bfff640 1 -- 172.21.15.102:0/3894206384 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3800cbc0 con 0x7f1f4412c900 2024-09-13T17:42:25.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.353+0000 7f1f4a75d640 1 -- 172.21.15.102:0/3894206384 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f38009390 con 0x7f1f4412c900 2024-09-13T17:42:25.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.354+0000 7f1f2bfff640 1 -- 172.21.15.102:0/3894206384 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f38013470 con 0x7f1f4412c900 2024-09-13T17:42:25.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.354+0000 7f1f4a75d640 1 -- 172.21.15.102:0/3894206384 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3c009a20 con 0x7f1f4412c900 2024-09-13T17:42:25.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.354+0000 7f1f2bfff640 1 -- 172.21.15.102:0/3894206384 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f1f380137e0 con 0x7f1f4412c900 2024-09-13T17:42:25.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.354+0000 7f1f2bfff640 1 -- 172.21.15.102:0/3894206384 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3660+0+0 (secure 0 0 0) 0x7f1f38042410 con 0x7f1f4412c900 2024-09-13T17:42:25.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.355+0000 7f1f4a75d640 1 -- 172.21.15.102:0/3894206384 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f4411ce30 con 0x7f1f4412c900 2024-09-13T17:42:25.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.361+0000 7f1f2bfff640 1 -- 172.21.15.102:0/3894206384 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f1f38021080 con 0x7f1f4412c900 2024-09-13T17:42:25.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.515+0000 7f1f4a75d640 1 -- 172.21.15.102:0/3894206384 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1f44064000 con 0x7f1f4412c900 2024-09-13T17:42:25.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.515+0000 7f1f2bfff640 1 -- 172.21.15.102:0/3894206384 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1f38023ad0 con 0x7f1f4412c900 2024-09-13T17:42:25.518 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-09-13T17:42:25.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.517+0000 7f1f4a75d640 1 -- 172.21.15.102:0/3894206384 >> v1:172.21.15.102:6816/34937 conn(0x7f1f30031160 legacy=0x7f1f30033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:25.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.517+0000 7f1f4a75d640 1 -- 172.21.15.102:0/3894206384 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1f4412c900 msgr2=0x7f1f4411ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:25.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.517+0000 7f1f4a75d640 1 --2- 172.21.15.102:0/3894206384 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1f4412c900 0x7f1f4411ad60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1f3800ec80 tx=0x7f1f380090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.517+0000 7f1f4a75d640 1 -- 172.21.15.102:0/3894206384 shutdown_connections 2024-09-13T17:42:25.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.517+0000 7f1f4a75d640 1 --2- 172.21.15.102:0/3894206384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f4412ccd0 0x7f1f4413c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.517+0000 7f1f4a75d640 1 --2- 172.21.15.102:0/3894206384 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1f4412c900 0x7f1f4411ad60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.517+0000 7f1f4a75d640 1 --2- 172.21.15.102:0/3894206384 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1f44096a70 0x7f1f4411a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.517+0000 7f1f4a75d640 1 -- 172.21.15.102:0/3894206384 >> 172.21.15.102:0/3894206384 conn(0x7f1f4409e6b0 msgr2=0x7f1f4412ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:25.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.517+0000 7f1f4a75d640 1 -- 172.21.15.102:0/3894206384 shutdown_connections 2024-09-13T17:42:25.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.517+0000 7f1f4a75d640 1 -- 172.21.15.102:0/3894206384 wait complete. 2024-09-13T17:42:25.530 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:42:25.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.625+0000 7fd6b9cc9640 1 Processor -- start 2024-09-13T17:42:25.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.625+0000 7fd6b9cc9640 1 -- start start 2024-09-13T17:42:25.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.626+0000 7fd6b9cc9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd6b4098ed0 0x7fd6b4131610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.626+0000 7fd6b9cc9640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd6b4131b50 0x7fd6b4133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.626+0000 7fd6b9cc9640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6b40999e0 con 0x7fd6b4096a70 2024-09-13T17:42:25.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.626+0000 7fd6b9cc9640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b4069c30 con 0x7fd6b4131b50 2024-09-13T17:42:25.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.626+0000 7fd6b9cc9640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b406ed00 con 0x7fd6b4098ed0 2024-09-13T17:42:25.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.626+0000 7fd6b37fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fd6b4096a70 0x7fd6b40987c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:47408/0 (socket says 172.21.15.102:47408) 2024-09-13T17:42:25.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.626+0000 7fd6b2ffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd6b4098ed0 0x7fd6b4131610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.626+0000 7fd6b37fe640 1 -- 172.21.15.102:0/3525572219 learned_addr learned my addr 172.21.15.102:0/3525572219 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:42:25.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.626+0000 7fd6b3fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd6b4131b50 0x7fd6b4133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.626+0000 7fd6b2ffd640 1 -- 172.21.15.102:0/3525572219 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd6b4131b50 msgr2=0x7fd6b4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:25.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.626+0000 7fd6b27fc640 1 -- 172.21.15.102:0/3525572219 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2398079645 0 0) 0x7fd6b40999e0 con 0x7fd6b4096a70 2024-09-13T17:42:25.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.626+0000 7fd6b2ffd640 1 --2- 172.21.15.102:0/3525572219 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd6b4131b50 0x7fd6b4133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.626+0000 7fd6b2ffd640 1 -- 172.21.15.102:0/3525572219 >> v1:172.21.15.102:6789/0 conn(0x7fd6b4096a70 legacy=0x7fd6b40987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:25.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.626+0000 7fd6b2ffd640 1 -- 172.21.15.102:0/3525572219 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b40997a0 con 0x7fd6b4098ed0 2024-09-13T17:42:25.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.626+0000 7fd6b3fff640 1 --2- 172.21.15.102:0/3525572219 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd6b4131b50 0x7fd6b4133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:25.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.627+0000 7fd6b2ffd640 1 --2- 172.21.15.102:0/3525572219 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd6b4098ed0 0x7fd6b4131610 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8009880 tx=0x7fd6a80090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b96b6d82921a3a9b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:25.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.627+0000 7fd6b27fc640 1 -- 172.21.15.102:0/3525572219 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a802acd0 con 0x7fd6b4098ed0 2024-09-13T17:42:25.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.627+0000 7fd6b27fc640 1 -- 172.21.15.102:0/3525572219 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a802ae30 con 0x7fd6b4098ed0 2024-09-13T17:42:25.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.627+0000 7fd6b27fc640 1 -- 172.21.15.102:0/3525572219 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8032670 con 0x7fd6b4098ed0 2024-09-13T17:42:25.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.627+0000 7fd6b9cc9640 1 -- 172.21.15.102:0/3525572219 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd6b4098ed0 msgr2=0x7fd6b4131610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:25.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.627+0000 7fd6b9cc9640 1 --2- 172.21.15.102:0/3525572219 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd6b4098ed0 0x7fd6b4131610 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8009880 tx=0x7fd6a80090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.627+0000 7fd6b9cc9640 1 -- 172.21.15.102:0/3525572219 shutdown_connections 2024-09-13T17:42:25.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.627+0000 7fd6b9cc9640 1 --2- 172.21.15.102:0/3525572219 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd6b4131b50 0x7fd6b4133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.627+0000 7fd6b9cc9640 1 --2- 172.21.15.102:0/3525572219 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd6b4098ed0 0x7fd6b4131610 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.627+0000 7fd6b9cc9640 1 -- 172.21.15.102:0/3525572219 >> 172.21.15.102:0/3525572219 conn(0x7fd6b409e720 msgr2=0x7fd6b409eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:25.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.628+0000 7fd6b9cc9640 1 -- 172.21.15.102:0/3525572219 shutdown_connections 2024-09-13T17:42:25.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.628+0000 7fd6b9cc9640 1 -- 172.21.15.102:0/3525572219 wait complete. 2024-09-13T17:42:25.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.628+0000 7fd6b9cc9640 1 Processor -- start 2024-09-13T17:42:25.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.628+0000 7fd6b9cc9640 1 -- start start 2024-09-13T17:42:25.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.629+0000 7fd6b9cc9640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd6b4096a70 0x7fd6b412ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.629+0000 7fd6b9cc9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd6b4098ed0 0x7fd6b41304f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.629+0000 7fd6b9cc9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6b4131b50 0x7fd6b4130a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.629+0000 7fd6b9cc9640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b4097790 con 0x7fd6b4131b50 2024-09-13T17:42:25.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.629+0000 7fd6b9cc9640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b4118390 con 0x7fd6b4096a70 2024-09-13T17:42:25.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.629+0000 7fd6b9cc9640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b4098130 con 0x7fd6b4098ed0 2024-09-13T17:42:25.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.629+0000 7fd6b37fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd6b4096a70 0x7fd6b412ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.629+0000 7fd6b2ffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd6b4098ed0 0x7fd6b41304f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.629+0000 7fd6b37fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd6b4096a70 0x7fd6b412ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:39748/0 (socket says 172.21.15.102:39748) 2024-09-13T17:42:25.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.629+0000 7fd6b2ffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd6b4098ed0 0x7fd6b41304f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:49002/0 (socket says 172.21.15.102:49002) 2024-09-13T17:42:25.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.629+0000 7fd6b37fe640 1 -- 172.21.15.102:0/2416506258 learned_addr learned my addr 172.21.15.102:0/2416506258 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:25.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.629+0000 7fd6b3fff640 1 --2- 172.21.15.102:0/2416506258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6b4131b50 0x7fd6b4130a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.629+0000 7fd6b2ffd640 1 -- 172.21.15.102:0/2416506258 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd6b4096a70 msgr2=0x7fd6b412ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:25.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.629+0000 7fd6b2ffd640 1 --2- 172.21.15.102:0/2416506258 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd6b4096a70 0x7fd6b412ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.629+0000 7fd6b2ffd640 1 -- 172.21.15.102:0/2416506258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6b4131b50 msgr2=0x7fd6b4130a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:25.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.629+0000 7fd6b2ffd640 1 --2- 172.21.15.102:0/2416506258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6b4131b50 0x7fd6b4130a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.629+0000 7fd6b2ffd640 1 -- 172.21.15.102:0/2416506258 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ac009a20 con 0x7fd6b4098ed0 2024-09-13T17:42:25.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.629+0000 7fd6b37fe640 1 --2- 172.21.15.102:0/2416506258 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd6b4096a70 0x7fd6b412ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:25.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.629+0000 7fd6b3fff640 1 --2- 172.21.15.102:0/2416506258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6b4131b50 0x7fd6b4130a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:25.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.629+0000 7fd6b2ffd640 1 --2- 172.21.15.102:0/2416506258 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd6b4098ed0 0x7fd6b41304f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4099730 tx=0x7fd6a8032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:25.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.630+0000 7fd69b7fe640 1 -- 172.21.15.102:0/2416506258 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8031700 con 0x7fd6b4098ed0 2024-09-13T17:42:25.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.630+0000 7fd69b7fe640 1 -- 172.21.15.102:0/2416506258 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a8031860 con 0x7fd6b4098ed0 2024-09-13T17:42:25.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.630+0000 7fd6b9cc9640 1 -- 172.21.15.102:0/2416506258 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6a8009390 con 0x7fd6b4098ed0 2024-09-13T17:42:25.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.630+0000 7fd69b7fe640 1 -- 172.21.15.102:0/2416506258 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8031c00 con 0x7fd6b4098ed0 2024-09-13T17:42:25.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.630+0000 7fd6b9cc9640 1 -- 172.21.15.102:0/2416506258 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6b4117150 con 0x7fd6b4098ed0 2024-09-13T17:42:25.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.630+0000 7fd69b7fe640 1 -- 172.21.15.102:0/2416506258 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fd6a8039040 con 0x7fd6b4098ed0 2024-09-13T17:42:25.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.631+0000 7fd69b7fe640 1 -- 172.21.15.102:0/2416506258 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3660+0+0 (secure 0 0 0) 0x7fd6a8061ac0 con 0x7fd6b4098ed0 2024-09-13T17:42:25.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.631+0000 7fd6b9cc9640 1 -- 172.21.15.102:0/2416506258 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6b409a410 con 0x7fd6b4098ed0 2024-09-13T17:42:25.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.635+0000 7fd69b7fe640 1 -- 172.21.15.102:0/2416506258 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fd6a803b400 con 0x7fd6b4098ed0 2024-09-13T17:42:25.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.787+0000 7fd6b9cc9640 1 -- 172.21.15.102:0/2416506258 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd6b4097c50 con 0x7fd6b4098ed0 2024-09-13T17:42:25.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.788+0000 7fd69b7fe640 1 -- 172.21.15.102:0/2416506258 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd6a8037070 con 0x7fd6b4098ed0 2024-09-13T17:42:25.794 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-09-13T17:42:25.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.790+0000 7fd6b9cc9640 1 -- 172.21.15.102:0/2416506258 >> v1:172.21.15.102:6816/34937 conn(0x7fd6a0031160 legacy=0x7fd6a0033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:25.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.790+0000 7fd6b9cc9640 1 -- 172.21.15.102:0/2416506258 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd6b4098ed0 msgr2=0x7fd6b41304f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:25.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.790+0000 7fd6b9cc9640 1 --2- 172.21.15.102:0/2416506258 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd6b4098ed0 0x7fd6b41304f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4099730 tx=0x7fd6a8032000 comp rx=0 tx=0).stop 2024-09-13T17:42:25.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.790+0000 7fd6b9cc9640 1 -- 172.21.15.102:0/2416506258 shutdown_connections 2024-09-13T17:42:25.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.790+0000 7fd6b9cc9640 1 --2- 172.21.15.102:0/2416506258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6b4131b50 0x7fd6b4130a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.790+0000 7fd6b9cc9640 1 --2- 172.21.15.102:0/2416506258 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd6b4098ed0 0x7fd6b41304f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.790+0000 7fd6b9cc9640 1 --2- 172.21.15.102:0/2416506258 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd6b4096a70 0x7fd6b412ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.790+0000 7fd6b9cc9640 1 -- 172.21.15.102:0/2416506258 >> 172.21.15.102:0/2416506258 conn(0x7fd6b409e720 msgr2=0x7fd6b409dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:25.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.790+0000 7fd6b9cc9640 1 -- 172.21.15.102:0/2416506258 shutdown_connections 2024-09-13T17:42:25.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.790+0000 7fd6b9cc9640 1 -- 172.21.15.102:0/2416506258 wait complete. 2024-09-13T17:42:25.802 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:42:25.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.894+0000 7f226fa2b640 1 Processor -- start 2024-09-13T17:42:25.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.894+0000 7f226fa2b640 1 -- start start 2024-09-13T17:42:25.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.895+0000 7f226fa2b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2268068920 0x7f226809d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.895+0000 7f226fa2b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f226809dc50 0x7f226809e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.895+0000 7f226fa2b640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f226812f310 0x7f226812f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.895+0000 7f226fa2b640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2268068cf0 con 0x7f226812f310 2024-09-13T17:42:25.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.895+0000 7f226fa2b640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22680695a0 con 0x7f2268068920 2024-09-13T17:42:25.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.895+0000 7f226fa2b640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f226809f090 con 0x7f226809dc50 2024-09-13T17:42:25.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.895+0000 7f226dfc8640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f226812f310 0x7f226812f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.895+0000 7f226cfc6640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f226809dc50 0x7f226809e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.895+0000 7f226d7c7640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2268068920 0x7f226809d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.895+0000 7f226dfc8640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f226812f310 0x7f226812f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58164/0 (socket says 172.21.15.102:58164) 2024-09-13T17:42:25.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.895+0000 7f226cfc6640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f226809dc50 0x7f226809e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:49012/0 (socket says 172.21.15.102:49012) 2024-09-13T17:42:25.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.895+0000 7f226dfc8640 1 -- 172.21.15.102:0/3061794845 learned_addr learned my addr 172.21.15.102:0/3061794845 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:25.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.895+0000 7f226d7c7640 1 --2- 172.21.15.102:0/3061794845 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2268068920 0x7f226809d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:39754/0 (socket says 172.21.15.102:39754) 2024-09-13T17:42:25.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.895+0000 7f226cfc6640 1 -- 172.21.15.102:0/3061794845 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2268068920 msgr2=0x7f226809d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:25.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.895+0000 7f226cfc6640 1 --2- 172.21.15.102:0/3061794845 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2268068920 0x7f226809d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.895+0000 7f226cfc6640 1 -- 172.21.15.102:0/3061794845 >> v2:172.21.15.102:3300/0 conn(0x7f226812f310 msgr2=0x7f226812f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:42:25.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.895+0000 7f226cfc6640 1 --2- 172.21.15.102:0/3061794845 >> v2:172.21.15.102:3300/0 conn(0x7f226812f310 0x7f226812f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.895+0000 7f226cfc6640 1 -- 172.21.15.102:0/3061794845 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f226809cce0 con 0x7f226809dc50 2024-09-13T17:42:25.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.895+0000 7f226dfc8640 1 --2- 172.21.15.102:0/3061794845 >> v2:172.21.15.102:3300/0 conn(0x7f226812f310 0x7f226812f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:25.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.895+0000 7f226d7c7640 1 --2- 172.21.15.102:0/3061794845 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2268068920 0x7f226809d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:25.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.896+0000 7f226cfc6640 1 --2- 172.21.15.102:0/3061794845 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f226809dc50 0x7f226809e020 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2258009850 tx=0x7f22580090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa7fc017c83da206 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:25.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.896+0000 7f225ffff640 1 -- 172.21.15.102:0/3061794845 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225802ac50 con 0x7f226809dc50 2024-09-13T17:42:25.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.896+0000 7f225ffff640 1 -- 172.21.15.102:0/3061794845 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f225802adb0 con 0x7f226809dc50 2024-09-13T17:42:25.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.896+0000 7f225ffff640 1 -- 172.21.15.102:0/3061794845 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2258032670 con 0x7f226809dc50 2024-09-13T17:42:25.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.897+0000 7f226fa2b640 1 -- 172.21.15.102:0/3061794845 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f226809dc50 msgr2=0x7f226809e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:25.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.897+0000 7f226fa2b640 1 --2- 172.21.15.102:0/3061794845 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f226809dc50 0x7f226809e020 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2258009850 tx=0x7f22580090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.897+0000 7f226fa2b640 1 -- 172.21.15.102:0/3061794845 shutdown_connections 2024-09-13T17:42:25.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.897+0000 7f226fa2b640 1 --2- 172.21.15.102:0/3061794845 >> v2:172.21.15.102:3300/0 conn(0x7f226812f310 0x7f226812f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.897+0000 7f226fa2b640 1 --2- 172.21.15.102:0/3061794845 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f226809dc50 0x7f226809e020 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.897+0000 7f226fa2b640 1 --2- 172.21.15.102:0/3061794845 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2268068920 0x7f226809d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.897+0000 7f226fa2b640 1 -- 172.21.15.102:0/3061794845 >> 172.21.15.102:0/3061794845 conn(0x7f226806f320 msgr2=0x7f22680a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:25.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.897+0000 7f226fa2b640 1 -- 172.21.15.102:0/3061794845 shutdown_connections 2024-09-13T17:42:25.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.897+0000 7f226fa2b640 1 -- 172.21.15.102:0/3061794845 wait complete. 2024-09-13T17:42:25.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.898+0000 7f226fa2b640 1 Processor -- start 2024-09-13T17:42:25.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.898+0000 7f226fa2b640 1 -- start start 2024-09-13T17:42:25.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.898+0000 7f226fa2b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2268068920 0x7f226811dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.898+0000 7f226fa2b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f226809dc50 0x7f226811e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.898+0000 7f226fa2b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f226812f310 0x7f226811e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.898+0000 7f226fa2b640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226809e620 con 0x7f226812f310 2024-09-13T17:42:25.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.898+0000 7f226fa2b640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22681210e0 con 0x7f226809dc50 2024-09-13T17:42:25.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.898+0000 7f226fa2b640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2268121680 con 0x7f2268068920 2024-09-13T17:42:25.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.898+0000 7f226d7c7640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2268068920 0x7f226811dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.898+0000 7f226cfc6640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f226809dc50 0x7f226811e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.898+0000 7f226d7c7640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2268068920 0x7f226811dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:49018/0 (socket says 172.21.15.102:49018) 2024-09-13T17:42:25.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.899+0000 7f226d7c7640 1 -- 172.21.15.102:0/1707815578 learned_addr learned my addr 172.21.15.102:0/1707815578 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:42:25.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.899+0000 7f226cfc6640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f226809dc50 0x7f226811e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:39766/0 (socket says 172.21.15.102:39766) 2024-09-13T17:42:25.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.899+0000 7f226dfc8640 1 --2- 172.21.15.102:0/1707815578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f226812f310 0x7f226811e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.899+0000 7f226d7c7640 1 -- 172.21.15.102:0/1707815578 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f226809dc50 msgr2=0x7f226811e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:25.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.899+0000 7f226d7c7640 1 --2- 172.21.15.102:0/1707815578 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f226809dc50 0x7f226811e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.899+0000 7f226d7c7640 1 -- 172.21.15.102:0/1707815578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f226812f310 msgr2=0x7f226811e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:25.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.899+0000 7f226d7c7640 1 --2- 172.21.15.102:0/1707815578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f226812f310 0x7f226811e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.899+0000 7f226d7c7640 1 -- 172.21.15.102:0/1707815578 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2264009390 con 0x7f2268068920 2024-09-13T17:42:25.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.899+0000 7f226dfc8640 1 --2- 172.21.15.102:0/1707815578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f226812f310 0x7f226811e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:42:25.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.899+0000 7f226d7c7640 1 --2- 172.21.15.102:0/1707815578 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2268068920 0x7f226811dca0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f226400ecb0 tx=0x7f22640090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:25.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.899+0000 7f225d7fa640 1 -- 172.21.15.102:0/1707815578 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226400cbc0 con 0x7f2268068920 2024-09-13T17:42:25.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.899+0000 7f225d7fa640 1 -- 172.21.15.102:0/1707815578 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226400cd20 con 0x7f2268068920 2024-09-13T17:42:25.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.899+0000 7f226fa2b640 1 -- 172.21.15.102:0/1707815578 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2260009ae0 con 0x7f2268068920 2024-09-13T17:42:25.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.899+0000 7f225d7fa640 1 -- 172.21.15.102:0/1707815578 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2264013680 con 0x7f2268068920 2024-09-13T17:42:25.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.899+0000 7f226fa2b640 1 -- 172.21.15.102:0/1707815578 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2258009390 con 0x7f2268068920 2024-09-13T17:42:25.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.900+0000 7f225d7fa640 1 -- 172.21.15.102:0/1707815578 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f22640139f0 con 0x7f2268068920 2024-09-13T17:42:25.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.900+0000 7f225d7fa640 1 -- 172.21.15.102:0/1707815578 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3660+0+0 (secure 0 0 0) 0x7f2264042460 con 0x7f2268068920 2024-09-13T17:42:25.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.900+0000 7f226fa2b640 1 -- 172.21.15.102:0/1707815578 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2268121300 con 0x7f2268068920 2024-09-13T17:42:25.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:25.905+0000 7f225d7fa640 1 -- 172.21.15.102:0/1707815578 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f2264021020 con 0x7f2268068920 2024-09-13T17:42:26.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:26.064+0000 7f226fa2b640 1 -- 172.21.15.102:0/1707815578 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2268064000 con 0x7f2268068920 2024-09-13T17:42:26.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:26.065+0000 7f225d7fa640 1 -- 172.21.15.102:0/1707815578 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f22640236b0 con 0x7f2268068920 2024-09-13T17:42:26.067 INFO:teuthology.orchestra.run.smithi102.stderr:listed 0 entries 2024-09-13T17:42:26.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:26.067+0000 7f226fa2b640 1 -- 172.21.15.102:0/1707815578 >> v1:172.21.15.102:6816/34937 conn(0x7f224c031160 legacy=0x7f224c033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:26.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:26.067+0000 7f226fa2b640 1 -- 172.21.15.102:0/1707815578 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2268068920 msgr2=0x7f226811dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:26.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:26.067+0000 7f226fa2b640 1 --2- 172.21.15.102:0/1707815578 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2268068920 0x7f226811dca0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f226400ecb0 tx=0x7f22640090f0 comp rx=0 tx=0).stop 2024-09-13T17:42:26.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:26.067+0000 7f226fa2b640 1 -- 172.21.15.102:0/1707815578 shutdown_connections 2024-09-13T17:42:26.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:26.067+0000 7f226fa2b640 1 --2- 172.21.15.102:0/1707815578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f226812f310 0x7f226811e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:26.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:26.067+0000 7f226fa2b640 1 --2- 172.21.15.102:0/1707815578 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f226809dc50 0x7f226811e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:26.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:26.067+0000 7f226fa2b640 1 --2- 172.21.15.102:0/1707815578 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2268068920 0x7f226811dca0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:26.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:26.067+0000 7f226fa2b640 1 -- 172.21.15.102:0/1707815578 >> 172.21.15.102:0/1707815578 conn(0x7f226806f320 msgr2=0x7f226809afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:26.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:26.067+0000 7f226fa2b640 1 -- 172.21.15.102:0/1707815578 shutdown_connections 2024-09-13T17:42:26.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:42:26.067+0000 7f226fa2b640 1 -- 172.21.15.102:0/1707815578 wait complete. 2024-09-13T17:42:26.079 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi171.front.sepia.ceph.com... 2024-09-13T17:42:26.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:26.079 DEBUG:teuthology.orchestra.run.smithi171:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-13T17:42:26.108 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi171.stderr:ceph-fuse[29183]: fuse finished with error 0 and tester_r 0 2024-09-13T17:42:26.110 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:42:26.127 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi171.front.sepia.ceph.com 2024-09-13T17:42:26.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:26.127 DEBUG:teuthology.orchestra.run.smithi171:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-13T17:42:26.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:26.205 DEBUG:teuthology.orchestra.run.smithi171:> ip netns list 2024-09-13T17:42:26.263 INFO:teuthology.orchestra.run.smithi171.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T17:42:26.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:26.264 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T17:42:26.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:26.333 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip link delete ceph-brx 2024-09-13T17:42:26.431 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-13T17:42:26.431 INFO:teuthology.run_tasks:Running task print... 2024-09-13T17:42:26.446 INFO:teuthology.task.print:**** done unmount client.0 2024-09-13T17:42:26.446 INFO:teuthology.run_tasks:Running task install.upgrade... 2024-09-13T17:42:26.581 INFO:teuthology.task.install:project ceph config {'client.0': None} overrides {'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'} 2024-09-13T17:42:26.581 INFO:teuthology.task.install:remote ubuntu@smithi171.front.sepia.ceph.com config {'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'} 2024-09-13T17:42:26.581 INFO:teuthology.task.install:extra packages: [] 2024-09-13T17:42:26.586 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-13T17:42:26.587 INFO:teuthology.task.install:Upgrading ceph rpm packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd 2024-09-13T17:42:26.587 DEBUG:teuthology.orchestra.run.smithi171:> rpm -q ceph-common --qf '%{VERSION}-%{RELEASE}' 2024-09-13T17:42:26.614 INFO:teuthology.orchestra.run.smithi171.stdout:17.2.7-1627.gbbdadfa3.el9 2024-09-13T17:42:26.614 INFO:teuthology.packaging:The installed version of ceph-common is 17.2.7-1627.gbbdadfa3.el9 2024-09-13T17:42:26.614 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:42:26.781 INFO:teuthology.task.install:Ceph rpm upgrade from 17.2.7-1627.gbbdadfa3.el9 to 18.2.4-860.g6c6b083f 2024-09-13T17:42:26.782 INFO:teuthology.task.install.rpm:Host smithi171 is: centos 9.stream x86_64 2024-09-13T17:42:26.782 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:42:26.936 INFO:teuthology.task.install.rpm:Repo base URL: https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/ 2024-09-13T17:42:26.936 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f /etc/yum.repos.d/ceph.repo 2024-09-13T17:42:27.090 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-13T17:42:27.090 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T17:42:27.090 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-13T17:42:27.118 DEBUG:teuthology.orchestra.run.smithi171:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-13T17:42:27.187 DEBUG:teuthology.orchestra.run.smithi171:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-13T17:42:27.254 DEBUG:teuthology.orchestra.run.smithi171:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-13T17:42:27.310 INFO:teuthology.orchestra.run.smithi171.stdout:check_obsoletes = 1 2024-09-13T17:42:27.322 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum clean all 2024-09-13T17:42:27.554 INFO:teuthology.orchestra.run.smithi171.stdout:60 files removed 2024-09-13T17:42:27.582 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y upgrade ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd 2024-09-13T17:42:28.158 INFO:teuthology.orchestra.run.smithi171.stdout:ceph packages for x86_64 277 kB/s | 78 kB 00:00 2024-09-13T17:42:28.442 INFO:teuthology.orchestra.run.smithi171.stdout:ceph noarch packages 48 kB/s | 11 kB 00:00 2024-09-13T17:42:28.718 INFO:teuthology.orchestra.run.smithi171.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-13T17:42:30.676 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.2 MB 00:01 2024-09-13T17:42:37.153 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - AppStream 3.8 MB/s | 20 MB 00:05 2024-09-13T17:42:41.421 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-13T17:42:42.830 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - Extras packages 92 kB/s | 19 kB 00:00 2024-09-13T17:42:44.385 INFO:teuthology.orchestra.run.smithi171.stdout:Extra Packages for Enterprise Linux 16 MB/s | 23 MB 00:01 2024-09-13T17:42:49.774 INFO:teuthology.orchestra.run.smithi171.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-13T17:42:52.036 INFO:teuthology.orchestra.run.smithi171.stderr:Package cephadm available, but not installed. 2024-09-13T17:42:52.037 INFO:teuthology.orchestra.run.smithi171.stdout:No match for argument: cephadm 2024-09-13T17:42:52.037 INFO:teuthology.orchestra.run.smithi171.stderr:Package ceph-immutable-object-cache available, but not installed. 2024-09-13T17:42:52.037 INFO:teuthology.orchestra.run.smithi171.stdout:No match for argument: ceph-immutable-object-cache 2024-09-13T17:42:52.048 INFO:teuthology.orchestra.run.smithi171.stderr:Package ceph-mgr-cephadm available, but not installed. 2024-09-13T17:42:52.048 INFO:teuthology.orchestra.run.smithi171.stdout:No match for argument: ceph-mgr-cephadm 2024-09-13T17:42:52.051 INFO:teuthology.orchestra.run.smithi171.stderr:Package ceph-volume available, but not installed. 2024-09-13T17:42:52.051 INFO:teuthology.orchestra.run.smithi171.stdout:No match for argument: ceph-volume 2024-09-13T17:42:52.142 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-13T17:42:52.145 INFO:teuthology.orchestra.run.smithi171.stdout:====================================================================================== 2024-09-13T17:42:52.146 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-09-13T17:42:52.146 INFO:teuthology.orchestra.run.smithi171.stdout:====================================================================================== 2024-09-13T17:42:52.146 INFO:teuthology.orchestra.run.smithi171.stdout:Upgrading: 2024-09-13T17:42:52.146 INFO:teuthology.orchestra.run.smithi171.stdout: ceph x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 6.5 k 2024-09-13T17:42:52.146 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-base x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 5.1 M 2024-09-13T17:42:52.146 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-common x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 18 M 2024-09-13T17:42:52.146 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-fuse x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 852 k 2024-09-13T17:42:52.146 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-grafana-dashboards noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 24 k 2024-09-13T17:42:52.146 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mds x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 2.1 M 2024-09-13T17:42:52.146 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 1.5 M 2024-09-13T17:42:52.146 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-dashboard noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 3.5 M 2024-09-13T17:42:52.146 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 7.4 M 2024-09-13T17:42:52.146 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-modules-core noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 246 k 2024-09-13T17:42:52.147 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-rook noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 49 k 2024-09-13T17:42:52.147 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mon x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 4.7 M 2024-09-13T17:42:52.147 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-osd x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 17 M 2024-09-13T17:42:52.147 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-prometheus-alerts noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 16 k 2024-09-13T17:42:52.147 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-radosgw x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 7.7 M 2024-09-13T17:42:52.147 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-selinux x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 25 k 2024-09-13T17:42:52.147 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-test x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 40 M 2024-09-13T17:42:52.147 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs-devel x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 31 k 2024-09-13T17:42:52.147 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs2 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 712 k 2024-09-13T17:42:52.147 INFO:teuthology.orchestra.run.smithi171.stdout: libcephsqlite x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 166 k 2024-09-13T17:42:52.147 INFO:teuthology.orchestra.run.smithi171.stdout: librados-devel x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 127 k 2024-09-13T17:42:52.147 INFO:teuthology.orchestra.run.smithi171.stdout: librados2 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 3.3 M 2024-09-13T17:42:52.148 INFO:teuthology.orchestra.run.smithi171.stdout: libradosstriper1 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 478 k 2024-09-13T17:42:52.148 INFO:teuthology.orchestra.run.smithi171.stdout: librbd1 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 3.0 M 2024-09-13T17:42:52.148 INFO:teuthology.orchestra.run.smithi171.stdout: librgw2 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 4.5 M 2024-09-13T17:42:52.148 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-argparse x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 45 k 2024-09-13T17:42:52.148 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-common x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 129 k 2024-09-13T17:42:52.148 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cephfs x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 162 k 2024-09-13T17:42:52.148 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rados x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 321 k 2024-09-13T17:42:52.148 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rbd x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 301 k 2024-09-13T17:42:52.148 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rgw x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 99 k 2024-09-13T17:42:52.148 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-fuse x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 87 k 2024-09-13T17:42:52.148 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-mirror x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 3.0 M 2024-09-13T17:42:52.148 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-nbd x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 172 k 2024-09-13T17:42:52.148 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-09-13T17:42:52.148 INFO:teuthology.orchestra.run.smithi171.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-13T17:42:52.149 INFO:teuthology.orchestra.run.smithi171.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-13T17:42:52.149 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-13T17:42:52.149 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-13T17:42:52.149 INFO:teuthology.orchestra.run.smithi171.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-13T17:42:52.149 INFO:teuthology.orchestra.run.smithi171.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-13T17:42:52.149 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:42:52.149 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-13T17:42:52.149 INFO:teuthology.orchestra.run.smithi171.stdout:====================================================================================== 2024-09-13T17:42:52.149 INFO:teuthology.orchestra.run.smithi171.stdout:Install 6 Packages 2024-09-13T17:42:52.149 INFO:teuthology.orchestra.run.smithi171.stdout:Upgrade 34 Packages 2024-09-13T17:42:52.149 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:42:52.150 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 130 M 2024-09-13T17:42:52.150 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-13T17:42:52.484 INFO:teuthology.orchestra.run.smithi171.stdout:(1/40): fmt-8.1.1-5.el9.x86_64.rpm 1.9 MB/s | 111 kB 00:00 2024-09-13T17:42:52.509 INFO:teuthology.orchestra.run.smithi171.stdout:(2/40): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.1 MB/s | 26 kB 00:00 2024-09-13T17:42:52.576 INFO:teuthology.orchestra.run.smithi171.stdout:(3/40): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-13T17:42:52.627 INFO:teuthology.orchestra.run.smithi171.stdout:(4/40): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-13T17:42:52.669 INFO:teuthology.orchestra.run.smithi171.stdout:(5/40): boost-program-options-1.75.0-8.el9.x86_ 440 kB/s | 107 kB 00:00 2024-09-13T17:42:52.769 INFO:teuthology.orchestra.run.smithi171.stdout:(6/40): libarrow-9.0.0-11.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-09-13T17:42:52.836 INFO:teuthology.orchestra.run.smithi171.stdout:(7/40): ceph-18.2.4-860.g6c6b083f.el9.x86_64.rp 31 kB/s | 6.5 kB 00:00 2024-09-13T17:42:53.145 INFO:teuthology.orchestra.run.smithi171.stdout:(8/40): ceph-fuse-18.2.4-860.g6c6b083f.el9.x86_ 2.7 MB/s | 852 kB 00:00 2024-09-13T17:42:53.503 INFO:teuthology.orchestra.run.smithi171.stdout:(9/40): ceph-mds-18.2.4-860.g6c6b083f.el9.x86_6 5.9 MB/s | 2.1 MB 00:00 2024-09-13T17:42:53.704 INFO:teuthology.orchestra.run.smithi171.stdout:(10/40): ceph-base-18.2.4-860.g6c6b083f.el9.x86 4.9 MB/s | 5.1 MB 00:01 2024-09-13T17:42:53.746 INFO:teuthology.orchestra.run.smithi171.stdout:(11/40): ceph-mgr-18.2.4-860.g6c6b083f.el9.x86_ 6.0 MB/s | 1.5 MB 00:00 2024-09-13T17:42:54.054 INFO:teuthology.orchestra.run.smithi171.stdout:(12/40): ceph-mon-18.2.4-860.g6c6b083f.el9.x86_ 13 MB/s | 4.7 MB 00:00 2024-09-13T17:42:54.663 INFO:teuthology.orchestra.run.smithi171.stdout:(13/40): ceph-common-18.2.4-860.g6c6b083f.el9.x 9.7 MB/s | 18 MB 00:01 2024-09-13T17:42:54.764 INFO:teuthology.orchestra.run.smithi171.stdout:(14/40): ceph-radosgw-18.2.4-860.g6c6b083f.el9. 11 MB/s | 7.7 MB 00:00 2024-09-13T17:42:54.789 INFO:teuthology.orchestra.run.smithi171.stdout:(15/40): ceph-selinux-18.2.4-860.g6c6b083f.el9. 200 kB/s | 25 kB 00:00 2024-09-13T17:42:54.831 INFO:teuthology.orchestra.run.smithi171.stdout:(16/40): libcephfs-devel-18.2.4-860.g6c6b083f.e 747 kB/s | 31 kB 00:00 2024-09-13T17:42:54.998 INFO:teuthology.orchestra.run.smithi171.stdout:(17/40): ceph-osd-18.2.4-860.g6c6b083f.el9.x86_ 13 MB/s | 17 MB 00:01 2024-09-13T17:42:55.032 INFO:teuthology.orchestra.run.smithi171.stdout:(18/40): libcephfs2-18.2.4-860.g6c6b083f.el9.x8 3.5 MB/s | 712 kB 00:00 2024-09-13T17:42:55.057 INFO:teuthology.orchestra.run.smithi171.stdout:(19/40): libcephsqlite-18.2.4-860.g6c6b083f.el9 2.8 MB/s | 166 kB 00:00 2024-09-13T17:42:55.083 INFO:teuthology.orchestra.run.smithi171.stdout:(20/40): librados-devel-18.2.4-860.g6c6b083f.el 2.5 MB/s | 127 kB 00:00 2024-09-13T17:42:55.150 INFO:teuthology.orchestra.run.smithi171.stdout:(21/40): libradosstriper1-18.2.4-860.g6c6b083f. 7.0 MB/s | 478 kB 00:00 2024-09-13T17:42:55.267 INFO:teuthology.orchestra.run.smithi171.stdout:(22/40): librados2-18.2.4-860.g6c6b083f.el9.x86 16 MB/s | 3.3 MB 00:00 2024-09-13T17:42:55.384 INFO:teuthology.orchestra.run.smithi171.stdout:(23/40): librbd1-18.2.4-860.g6c6b083f.el9.x86_6 13 MB/s | 3.0 MB 00:00 2024-09-13T17:42:55.451 INFO:teuthology.orchestra.run.smithi171.stdout:(24/40): python3-ceph-argparse-18.2.4-860.g6c6b 675 kB/s | 45 kB 00:00 2024-09-13T17:42:55.543 INFO:teuthology.orchestra.run.smithi171.stdout:(25/40): librgw2-18.2.4-860.g6c6b083f.el9.x86_6 16 MB/s | 4.5 MB 00:00 2024-09-13T17:42:55.569 INFO:teuthology.orchestra.run.smithi171.stdout:(26/40): python3-ceph-common-18.2.4-860.g6c6b08 1.1 MB/s | 129 kB 00:00 2024-09-13T17:42:55.594 INFO:teuthology.orchestra.run.smithi171.stdout:(27/40): python3-cephfs-18.2.4-860.g6c6b083f.el 3.1 MB/s | 162 kB 00:00 2024-09-13T17:42:55.627 INFO:teuthology.orchestra.run.smithi171.stdout:(28/40): python3-rados-18.2.4-860.g6c6b083f.el9 5.3 MB/s | 321 kB 00:00 2024-09-13T17:42:55.653 INFO:teuthology.orchestra.run.smithi171.stdout:(29/40): python3-rbd-18.2.4-860.g6c6b083f.el9.x 5.0 MB/s | 301 kB 00:00 2024-09-13T17:42:55.686 INFO:teuthology.orchestra.run.smithi171.stdout:(30/40): python3-rgw-18.2.4-860.g6c6b083f.el9.x 1.6 MB/s | 99 kB 00:00 2024-09-13T17:42:55.712 INFO:teuthology.orchestra.run.smithi171.stdout:(31/40): rbd-fuse-18.2.4-860.g6c6b083f.el9.x86_ 1.4 MB/s | 87 kB 00:00 2024-09-13T17:42:55.770 INFO:teuthology.orchestra.run.smithi171.stdout:(32/40): rbd-nbd-18.2.4-860.g6c6b083f.el9.x86_6 2.9 MB/s | 172 kB 00:00 2024-09-13T17:42:55.863 INFO:teuthology.orchestra.run.smithi171.stdout:(33/40): ceph-grafana-dashboards-18.2.4-860.g6c 266 kB/s | 24 kB 00:00 2024-09-13T17:42:55.921 INFO:teuthology.orchestra.run.smithi171.stdout:(34/40): rbd-mirror-18.2.4-860.g6c6b083f.el9.x8 13 MB/s | 3.0 MB 00:00 2024-09-13T17:42:56.155 INFO:teuthology.orchestra.run.smithi171.stdout:(35/40): ceph-mgr-dashboard-18.2.4-860.g6c6b083 12 MB/s | 3.5 MB 00:00 2024-09-13T17:42:56.223 INFO:teuthology.orchestra.run.smithi171.stdout:(36/40): ceph-mgr-modules-core-18.2.4-860.g6c6b 3.6 MB/s | 246 kB 00:00 2024-09-13T17:42:56.290 INFO:teuthology.orchestra.run.smithi171.stdout:(37/40): ceph-mgr-rook-18.2.4-860.g6c6b083f.el9 742 kB/s | 49 kB 00:00 2024-09-13T17:42:56.348 INFO:teuthology.orchestra.run.smithi171.stdout:(38/40): ceph-prometheus-alerts-18.2.4-860.g6c6 282 kB/s | 16 kB 00:00 2024-09-13T17:42:56.724 INFO:teuthology.orchestra.run.smithi171.stdout:(39/40): ceph-test-18.2.4-860.g6c6b083f.el9.x86 20 MB/s | 40 MB 00:01 2024-09-13T17:42:56.841 INFO:teuthology.orchestra.run.smithi171.stdout:(40/40): ceph-mgr-diskprediction-local-18.2.4-8 8.1 MB/s | 7.4 MB 00:00 2024-09-13T17:42:56.845 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-13T17:42:56.846 INFO:teuthology.orchestra.run.smithi171.stdout:Total 28 MB/s | 130 MB 00:04 2024-09-13T17:42:56.860 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-13T17:42:56.928 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-13T17:42:56.928 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-13T17:42:57.829 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-13T17:42:57.829 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-13T17:43:00.771 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-13T17:43:00.892 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 1/74 2024-09-13T17:43:01.240 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 1/74 2024-09-13T17:43:01.311 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 2/74 2024-09-13T17:43:01.434 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 2/74 2024-09-13T17:43:01.479 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 3/74 2024-09-13T17:43:01.571 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 3/74 2024-09-13T17:43:01.627 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : python3-rados-2:18.2.4-860.g6c6b083f.el9.x86_64 4/74 2024-09-13T17:43:01.664 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 5/74 2024-09-13T17:43:01.783 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 5/74 2024-09-13T17:43:01.824 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_64 6/74 2024-09-13T17:43:01.891 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_64 6/74 2024-09-13T17:43:01.942 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : python3-ceph-argparse-2:18.2.4-860.g6c6b083f.el9.x 7/74 2024-09-13T17:43:02.014 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : python3-cephfs-2:18.2.4-860.g6c6b083f.el9.x86_64 8/74 2024-09-13T17:43:02.082 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : python3-rbd-2:18.2.4-860.g6c6b083f.el9.x86_64 9/74 2024-09-13T17:43:02.122 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : librados-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 10/74 2024-09-13T17:43:02.158 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : ceph-prometheus-alerts-2:18.2.4-860.g6c6b083f.el9. 11/74 2024-09-13T17:43:02.240 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : ceph-grafana-dashboards-2:18.2.4-860.g6c6b083f.el9 12/74 2024-09-13T17:43:02.302 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : python3-ceph-common-2:18.2.4-860.g6c6b083f.el9.x86 13/74 2024-09-13T17:43:02.409 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : re2-1:20211101-20.el9.x86_64 14/74 2024-09-13T17:43:02.467 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 15/74 2024-09-13T17:43:02.511 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/74 2024-09-13T17:43:02.553 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/74 2024-09-13T17:43:02.589 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 18/74 2024-09-13T17:43:03.076 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 19/74 2024-09-13T17:43:03.127 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 20/74 2024-09-13T17:43:03.198 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 20/74 2024-09-13T17:43:03.245 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : python3-rgw-2:18.2.4-860.g6c6b083f.el9.x86_64 21/74 2024-09-13T17:43:05.428 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 22/74 2024-09-13T17:43:05.523 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 22/74 2024-09-13T17:43:06.165 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 22/74 2024-09-13T17:43:06.319 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 23/74 2024-09-13T17:43:06.384 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 23/74 2024-09-13T17:43:06.421 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 24/74 2024-09-13T17:43:16.212 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 24/74 2024-09-13T17:43:16.336 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.noar 25/74 2024-09-13T17:43:17.604 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.noar 25/74 2024-09-13T17:43:17.889 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b08 26/74 2024-09-13T17:43:18.016 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b08 26/74 2024-09-13T17:43:18.199 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : ceph-mgr-modules-core-2:18.2.4-860.g6c6b083f.el9.n 27/74 2024-09-13T17:43:18.292 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 28/74 2024-09-13T17:43:18.322 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 28/74 2024-09-13T17:43:18.376 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 29/74 2024-09-13T17:43:18.554 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 29/74 2024-09-13T17:43:18.635 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 30/74 2024-09-13T17:43:19.155 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 30/74 2024-09-13T17:43:19.236 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 31/74 2024-09-13T17:43:21.043 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 31/74 2024-09-13T17:43:21.105 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 32/74 2024-09-13T17:43:21.117 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 32/74 2024-09-13T17:43:22.012 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : ceph-2:18.2.4-860.g6c6b083f.el9.x86_64 33/74 2024-09-13T17:43:22.081 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 34/74 2024-09-13T17:43:22.417 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 34/74 2024-09-13T17:43:22.457 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 35/74 2024-09-13T17:43:27.090 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 35/74 2024-09-13T17:43:27.419 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : ceph-test-2:18.2.4-860.g6c6b083f.el9.x86_64 36/74 2024-09-13T17:43:27.472 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : libcephfs-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 37/74 2024-09-13T17:43:27.522 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : rbd-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 38/74 2024-09-13T17:43:27.638 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : rbd-nbd-2:18.2.4-860.g6c6b083f.el9.x86_64 39/74 2024-09-13T17:43:27.708 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : ceph-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 40/74 2024-09-13T17:43:27.739 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : ceph-2:17.2.7-1627.gbbdadfa3.el9.x86_64 41/74 2024-09-13T17:43:27.782 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : ceph-test-2:17.2.7-1627.gbbdadfa3.el9.x86_64 42/74 2024-09-13T17:43:27.782 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : ceph-mgr-rook-2:17.2.7-1627.gbbdadfa3.el9.noarch 43/74 2024-09-13T17:43:27.824 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1627.gbbdadfa3.el9.noarch 43/74 2024-09-13T17:43:27.829 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1627.gbbdadfa3.el9.x86_64 44/74 2024-09-13T17:43:27.829 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : ceph-mgr-2:17.2.7-1627.gbbdadfa3.el9.x86_64 44/74 2024-09-13T17:43:27.857 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1627.gbbdadfa3.el9.x86_64 44/74 2024-09-13T17:43:27.882 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : ceph-mgr-modules-core-2:17.2.7-1627.gbbdadfa3.el9. 45/74 2024-09-13T17:43:27.890 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1627.gbbdadfa3.el9.x86_64 46/74 2024-09-13T17:43:27.890 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : ceph-radosgw-2:17.2.7-1627.gbbdadfa3.el9.x86_64 46/74 2024-09-13T17:43:27.976 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1627.gbbdadfa3.el9.x86_64 46/74 2024-09-13T17:43:28.009 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : libcephfs-devel-2:17.2.7-1627.gbbdadfa3.el9.x86_64 47/74 2024-09-13T17:43:28.034 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : rbd-nbd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 48/74 2024-09-13T17:43:28.039 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1627.gbbdadfa3.el9.x86_64 49/74 2024-09-13T17:43:28.039 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : rbd-mirror-2:17.2.7-1627.gbbdadfa3.el9.x86_64 49/74 2024-09-13T17:43:28.077 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1627.gbbdadfa3.el9.x86_64 49/74 2024-09-13T17:43:28.112 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : rbd-fuse-2:17.2.7-1627.gbbdadfa3.el9.x86_64 50/74 2024-09-13T17:43:28.112 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : ceph-mgr-dashboard-2:17.2.7-1627.gbbdadfa3.el9.noa 51/74 2024-09-13T17:43:28.153 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1627.gbbdadfa3.el9.noa 51/74 2024-09-13T17:43:28.154 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : libcephsqlite-2:17.2.7-1627.gbbdadfa3.el9.x86_64 52/74 2024-09-13T17:43:28.221 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1627.gbbdadfa3.el9.x86_64 52/74 2024-09-13T17:43:28.245 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : librados-devel-2:17.2.7-1627.gbbdadfa3.el9.x86_64 53/74 2024-09-13T17:43:28.250 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mds-2:17.2.7-1627.gbbdadfa3.el9.x86_64 54/74 2024-09-13T17:43:28.250 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : ceph-mds-2:17.2.7-1627.gbbdadfa3.el9.x86_64 54/74 2024-09-13T17:43:28.279 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mds-2:17.2.7-1627.gbbdadfa3.el9.x86_64 54/74 2024-09-13T17:43:28.284 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mon-2:17.2.7-1627.gbbdadfa3.el9.x86_64 55/74 2024-09-13T17:43:28.284 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : ceph-mon-2:17.2.7-1627.gbbdadfa3.el9.x86_64 55/74 2024-09-13T17:43:28.321 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mon-2:17.2.7-1627.gbbdadfa3.el9.x86_64 55/74 2024-09-13T17:43:28.326 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-osd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 56/74 2024-09-13T17:43:28.326 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : ceph-osd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 56/74 2024-09-13T17:43:28.364 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-osd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 56/74 2024-09-13T17:43:28.364 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : ceph-selinux-2:17.2.7-1627.gbbdadfa3.el9.x86_64 57/74 2024-09-13T17:43:28.397 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1627.gbbdadfa3.el9.x86_64 57/74 2024-09-13T17:43:28.403 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-base-2:17.2.7-1627.gbbdadfa3.el9.x86_64 58/74 2024-09-13T17:43:28.404 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : ceph-base-2:17.2.7-1627.gbbdadfa3.el9.x86_64 58/74 2024-09-13T17:43:28.518 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-base-2:17.2.7-1627.gbbdadfa3.el9.x86_64 58/74 2024-09-13T17:43:28.518 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : ceph-common-2:17.2.7-1627.gbbdadfa3.el9.x86_64 59/74 2024-09-13T17:43:28.559 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-common-2:17.2.7-1627.gbbdadfa3.el9.x86_64 59/74 2024-09-13T17:43:28.559 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : libradosstriper1-2:17.2.7-1627.gbbdadfa3.el9.x86_6 60/74 2024-09-13T17:43:28.652 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1627.gbbdadfa3.el9.x86_6 60/74 2024-09-13T17:43:28.677 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : python3-cephfs-2:17.2.7-1627.gbbdadfa3.el9.x86_64 61/74 2024-09-13T17:43:28.703 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : python3-rbd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 62/74 2024-09-13T17:43:28.703 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : librbd1-2:17.2.7-1627.gbbdadfa3.el9.x86_64 63/74 2024-09-13T17:43:28.778 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librbd1-2:17.2.7-1627.gbbdadfa3.el9.x86_64 63/74 2024-09-13T17:43:28.803 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : python3-rgw-2:17.2.7-1627.gbbdadfa3.el9.x86_64 64/74 2024-09-13T17:43:28.803 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : librgw2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 65/74 2024-09-13T17:43:28.896 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librgw2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 65/74 2024-09-13T17:43:28.921 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : python3-rados-2:17.2.7-1627.gbbdadfa3.el9.x86_64 66/74 2024-09-13T17:43:28.921 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : libcephfs2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 67/74 2024-09-13T17:43:29.005 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephfs2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 67/74 2024-09-13T17:43:29.030 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : ceph-fuse-2:17.2.7-1627.gbbdadfa3.el9.x86_64 68/74 2024-09-13T17:43:29.064 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : python3-ceph-argparse-2:17.2.7-1627.gbbdadfa3.el9. 69/74 2024-09-13T17:43:29.098 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : python3-ceph-common-2:17.2.7-1627.gbbdadfa3.el9.x8 70/74 2024-09-13T17:43:29.123 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : ceph-grafana-dashboards-2:17.2.7-1627.gbbdadfa3.el 71/74 2024-09-13T17:43:29.148 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : ceph-prometheus-alerts-2:17.2.7-1627.gbbdadfa3.el9 72/74 2024-09-13T17:43:29.148 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : ceph-mgr-diskprediction-local-2:17.2.7-1627.gbbdad 73/74 2024-09-13T17:43:29.191 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1627.gbbdad 73/74 2024-09-13T17:43:29.191 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : librados2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 74/74 2024-09-13T17:43:31.074 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 74/74 2024-09-13T17:43:31.074 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 1/74 2024-09-13T17:43:31.074 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 2/74 2024-09-13T17:43:31.074 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 3/74 2024-09-13T17:43:31.074 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 4/74 2024-09-13T17:43:31.075 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 5/74 2024-09-13T17:43:31.075 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : re2-1:20211101-20.el9.x86_64 6/74 2024-09-13T17:43:31.075 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-2:18.2.4-860.g6c6b083f.el9.x86_64 7/74 2024-09-13T17:43:31.075 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-2:17.2.7-1627.gbbdadfa3.el9.x86_64 8/74 2024-09-13T17:43:31.075 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 9/74 2024-09-13T17:43:31.075 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-base-2:17.2.7-1627.gbbdadfa3.el9.x86_64 10/74 2024-09-13T17:43:31.075 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 11/74 2024-09-13T17:43:31.075 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-common-2:17.2.7-1627.gbbdadfa3.el9.x86_64 12/74 2024-09-13T17:43:31.075 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 13/74 2024-09-13T17:43:31.075 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-fuse-2:17.2.7-1627.gbbdadfa3.el9.x86_64 14/74 2024-09-13T17:43:31.075 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 15/74 2024-09-13T17:43:31.076 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mds-2:17.2.7-1627.gbbdadfa3.el9.x86_64 16/74 2024-09-13T17:43:31.076 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 17/74 2024-09-13T17:43:31.076 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-2:17.2.7-1627.gbbdadfa3.el9.x86_64 18/74 2024-09-13T17:43:31.076 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 19/74 2024-09-13T17:43:31.076 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mon-2:17.2.7-1627.gbbdadfa3.el9.x86_64 20/74 2024-09-13T17:43:31.076 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 21/74 2024-09-13T17:43:31.076 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-osd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 22/74 2024-09-13T17:43:31.076 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 23/74 2024-09-13T17:43:31.076 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-radosgw-2:17.2.7-1627.gbbdadfa3.el9.x86_64 24/74 2024-09-13T17:43:31.076 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 25/74 2024-09-13T17:43:31.076 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-selinux-2:17.2.7-1627.gbbdadfa3.el9.x86_64 26/74 2024-09-13T17:43:31.076 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-test-2:18.2.4-860.g6c6b083f.el9.x86_64 27/74 2024-09-13T17:43:31.077 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-test-2:17.2.7-1627.gbbdadfa3.el9.x86_64 28/74 2024-09-13T17:43:31.077 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephfs-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 29/74 2024-09-13T17:43:31.077 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephfs-devel-2:17.2.7-1627.gbbdadfa3.el9.x86_64 30/74 2024-09-13T17:43:31.077 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 31/74 2024-09-13T17:43:31.081 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephfs2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 32/74 2024-09-13T17:43:31.081 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 33/74 2024-09-13T17:43:31.081 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephsqlite-2:17.2.7-1627.gbbdadfa3.el9.x86_64 34/74 2024-09-13T17:43:31.081 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 35/74 2024-09-13T17:43:31.081 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados-devel-2:17.2.7-1627.gbbdadfa3.el9.x86_64 36/74 2024-09-13T17:43:31.081 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 37/74 2024-09-13T17:43:31.082 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 38/74 2024-09-13T17:43:31.082 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_64 39/74 2024-09-13T17:43:31.082 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libradosstriper1-2:17.2.7-1627.gbbdadfa3.el9.x86_6 40/74 2024-09-13T17:43:31.082 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 41/74 2024-09-13T17:43:31.082 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librbd1-2:17.2.7-1627.gbbdadfa3.el9.x86_64 42/74 2024-09-13T17:43:31.082 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 43/74 2024-09-13T17:43:31.082 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librgw2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 44/74 2024-09-13T17:43:31.082 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ceph-argparse-2:18.2.4-860.g6c6b083f.el9.x 45/74 2024-09-13T17:43:31.082 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1627.gbbdadfa3.el9. 46/74 2024-09-13T17:43:31.082 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ceph-common-2:18.2.4-860.g6c6b083f.el9.x86 47/74 2024-09-13T17:43:31.082 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ceph-common-2:17.2.7-1627.gbbdadfa3.el9.x8 48/74 2024-09-13T17:43:31.082 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cephfs-2:18.2.4-860.g6c6b083f.el9.x86_64 49/74 2024-09-13T17:43:31.082 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cephfs-2:17.2.7-1627.gbbdadfa3.el9.x86_64 50/74 2024-09-13T17:43:31.082 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rados-2:18.2.4-860.g6c6b083f.el9.x86_64 51/74 2024-09-13T17:43:31.083 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rados-2:17.2.7-1627.gbbdadfa3.el9.x86_64 52/74 2024-09-13T17:43:31.083 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rbd-2:18.2.4-860.g6c6b083f.el9.x86_64 53/74 2024-09-13T17:43:31.083 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rbd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 54/74 2024-09-13T17:43:31.083 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rgw-2:18.2.4-860.g6c6b083f.el9.x86_64 55/74 2024-09-13T17:43:31.083 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rgw-2:17.2.7-1627.gbbdadfa3.el9.x86_64 56/74 2024-09-13T17:43:31.083 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 57/74 2024-09-13T17:43:31.083 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-fuse-2:17.2.7-1627.gbbdadfa3.el9.x86_64 58/74 2024-09-13T17:43:31.083 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 59/74 2024-09-13T17:43:31.083 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-mirror-2:17.2.7-1627.gbbdadfa3.el9.x86_64 60/74 2024-09-13T17:43:31.083 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-nbd-2:18.2.4-860.g6c6b083f.el9.x86_64 61/74 2024-09-13T17:43:31.083 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-nbd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 62/74 2024-09-13T17:43:31.083 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-860.g6c6b083f.el9 63/74 2024-09-13T17:43:31.083 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1627.gbbdadfa3.el 64/74 2024-09-13T17:43:31.083 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.noar 65/74 2024-09-13T17:43:31.083 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1627.gbbdadfa3.el9.noa 66/74 2024-09-13T17:43:31.084 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b08 67/74 2024-09-13T17:43:31.084 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1627.gbbdad 68/74 2024-09-13T17:43:31.084 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-860.g6c6b083f.el9.n 69/74 2024-09-13T17:43:31.084 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1627.gbbdadfa3.el9. 70/74 2024-09-13T17:43:31.084 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 71/74 2024-09-13T17:43:31.084 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1627.gbbdadfa3.el9.noarch 72/74 2024-09-13T17:43:31.084 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-860.g6c6b083f.el9. 73/74 2024-09-13T17:43:32.137 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1627.gbbdadfa3.el9 74/74 2024-09-13T17:43:32.137 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:43:32.137 INFO:teuthology.orchestra.run.smithi171.stdout:Upgraded: 2024-09-13T17:43:32.137 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:43:32.138 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:43:32.138 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:43:32.138 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:43:32.138 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-grafana-dashboards-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:43:32.138 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:43:32.138 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:43:32.138 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:43:32.138 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:43:32.138 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-modules-core-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:43:32.138 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:43:32.138 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:43:32.138 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:43:32.138 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-prometheus-alerts-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T17:43:32.139 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:43:32.139 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:43:32.139 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-test-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:43:32.139 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:43:32.139 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:43:32.139 INFO:teuthology.orchestra.run.smithi171.stdout: libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:43:32.139 INFO:teuthology.orchestra.run.smithi171.stdout: librados-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:43:32.139 INFO:teuthology.orchestra.run.smithi171.stdout: librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:43:32.139 INFO:teuthology.orchestra.run.smithi171.stdout: libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:43:32.139 INFO:teuthology.orchestra.run.smithi171.stdout: librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:43:32.139 INFO:teuthology.orchestra.run.smithi171.stdout: librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:43:32.139 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-argparse-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:43:32.140 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:43:32.140 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cephfs-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:43:32.140 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rados-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:43:32.140 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rbd-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:43:32.140 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rgw-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:43:32.140 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:43:32.140 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:43:32.140 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-nbd-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T17:43:32.140 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-13T17:43:32.140 INFO:teuthology.orchestra.run.smithi171.stdout: boost-program-options-1.75.0-8.el9.x86_64 fmt-8.1.1-5.el9.x86_64 2024-09-13T17:43:32.140 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-9.0.0-11.el9.x86_64 libarrow-doc-9.0.0-11.el9.noarch 2024-09-13T17:43:32.141 INFO:teuthology.orchestra.run.smithi171.stdout: parquet-libs-9.0.0-11.el9.x86_64 re2-1:20211101-20.el9.x86_64 2024-09-13T17:43:32.141 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T17:43:32.141 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-13T17:43:32.489 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:43:32.649 DEBUG:teuthology.orchestra.run.smithi171:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-13T17:43:32.671 INFO:teuthology.orchestra.run.smithi171.stdout:18.2.4-860.g6c6b083f.el9 2024-09-13T17:43:32.672 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-860.g6c6b083f.el9 2024-09-13T17:43:32.672 INFO:teuthology.task.install:The correct ceph version 18.2.4-860.g6c6b083f is installed. 2024-09-13T17:43:32.674 INFO:teuthology.run_tasks:Running task print... 2024-09-13T17:43:32.683 INFO:teuthology.task.print:**** done install.upgrade on client.0 2024-09-13T17:43:32.683 INFO:teuthology.run_tasks:Running task ceph.healthy... 2024-09-13T17:43:32.776 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-13T17:43:32.776 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-13T17:43:32.776 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-13T17:43:32.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.883+0000 7f2287f5d640 1 Processor -- start 2024-09-13T17:43:32.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.883+0000 7f2287f5d640 1 -- start start 2024-09-13T17:43:32.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.883+0000 7f2287f5d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f228006b140 0x7f22800998c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:32.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.883+0000 7f2287f5d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f228009ac20 0x7f2280099e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:32.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.884+0000 7f2287f5d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f228006ee90 con 0x7f228009a340 2024-09-13T17:43:32.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.884+0000 7f2287f5d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f228006ac90 con 0x7f228009ac20 2024-09-13T17:43:32.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.884+0000 7f2287f5d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2280099510 con 0x7f228006b140 2024-09-13T17:43:32.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.884+0000 7f22854f8640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f228009ac20 0x7f2280099e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:32.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.884+0000 7f2285cf9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f228006b140 0x7f22800998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:32.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.884+0000 7f22864fa640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f228009a340 0x7f228012d280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:56204/0 (socket says 172.21.15.102:56204) 2024-09-13T17:43:32.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.884+0000 7f22854f8640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f228009ac20 0x7f2280099e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57330/0 (socket says 172.21.15.102:57330) 2024-09-13T17:43:32.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.884+0000 7f2285cf9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f228006b140 0x7f22800998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47074/0 (socket says 172.21.15.102:47074) 2024-09-13T17:43:32.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.884+0000 7f22864fa640 1 -- 172.21.15.102:0/298840643 learned_addr learned my addr 172.21.15.102:0/298840643 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:32.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.884+0000 7f22854f8640 1 -- 172.21.15.102:0/298840643 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f228006b140 msgr2=0x7f22800998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:32.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.884+0000 7f22854f8640 1 --2- 172.21.15.102:0/298840643 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f228006b140 0x7f22800998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:32.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.884+0000 7f22854f8640 1 -- 172.21.15.102:0/298840643 >> v1:172.21.15.102:6789/0 conn(0x7f228009a340 legacy=0x7f228012d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:32.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.884+0000 7f22854f8640 1 -- 172.21.15.102:0/298840643 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2280099680 con 0x7f228009ac20 2024-09-13T17:43:32.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.884+0000 7f2284cf7640 1 -- 172.21.15.102:0/298840643 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1316799078 0 0) 0x7f228006ee90 con 0x7f228009a340 2024-09-13T17:43:32.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.884+0000 7f2285cf9640 1 --2- 172.21.15.102:0/298840643 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f228006b140 0x7f22800998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:32.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.885+0000 7f22854f8640 1 --2- 172.21.15.102:0/298840643 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f228009ac20 0x7f2280099e00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2270009850 tx=0x7f22700090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320f5364501c878 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:32.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.885+0000 7f2284cf7640 1 -- 172.21.15.102:0/298840643 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227002ac60 con 0x7f228009ac20 2024-09-13T17:43:32.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.885+0000 7f2284cf7640 1 -- 172.21.15.102:0/298840643 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227002adc0 con 0x7f228009ac20 2024-09-13T17:43:32.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.885+0000 7f2284cf7640 1 -- 172.21.15.102:0/298840643 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2270032670 con 0x7f228009ac20 2024-09-13T17:43:32.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.885+0000 7f2287f5d640 1 -- 172.21.15.102:0/298840643 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f228009ac20 msgr2=0x7f2280099e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:32.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.885+0000 7f2287f5d640 1 --2- 172.21.15.102:0/298840643 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f228009ac20 0x7f2280099e00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2270009850 tx=0x7f22700090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:32.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.886+0000 7f2287f5d640 1 -- 172.21.15.102:0/298840643 shutdown_connections 2024-09-13T17:43:32.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.886+0000 7f2287f5d640 1 --2- 172.21.15.102:0/298840643 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f228009ac20 0x7f2280099e00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:32.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.886+0000 7f2287f5d640 1 --2- 172.21.15.102:0/298840643 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f228006b140 0x7f22800998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:32.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.886+0000 7f2287f5d640 1 -- 172.21.15.102:0/298840643 >> 172.21.15.102:0/298840643 conn(0x7f228009e820 msgr2=0x7f228009ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:32.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.886+0000 7f2287f5d640 1 -- 172.21.15.102:0/298840643 shutdown_connections 2024-09-13T17:43:32.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.886+0000 7f2287f5d640 1 -- 172.21.15.102:0/298840643 wait complete. 2024-09-13T17:43:32.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.886+0000 7f2287f5d640 1 Processor -- start 2024-09-13T17:43:32.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.886+0000 7f2287f5d640 1 -- start start 2024-09-13T17:43:32.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.887+0000 7f2287f5d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f228006b140 0x7f228011f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:32.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.887+0000 7f2287f5d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f228009a340 0x7f228011c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:32.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.887+0000 7f2287f5d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f228009ac20 0x7f228011c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:32.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.887+0000 7f2287f5d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228011d780 con 0x7f228006b140 2024-09-13T17:43:32.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.887+0000 7f2287f5d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f228006b930 con 0x7f228009a340 2024-09-13T17:43:32.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.887+0000 7f2287f5d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f228011bcb0 con 0x7f228009ac20 2024-09-13T17:43:32.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.887+0000 7f2285cf9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f228006b140 0x7f228011f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:32.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.887+0000 7f2285cf9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f228006b140 0x7f228011f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57336/0 (socket says 172.21.15.102:57336) 2024-09-13T17:43:32.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.887+0000 7f2285cf9640 1 -- 172.21.15.102:0/1156501557 learned_addr learned my addr 172.21.15.102:0/1156501557 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:32.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.887+0000 7f22864fa640 1 --2- 172.21.15.102:0/1156501557 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f228009ac20 0x7f228011c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:32.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.887+0000 7f22854f8640 1 --2- 172.21.15.102:0/1156501557 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f228009a340 0x7f228011c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:32.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.887+0000 7f2285cf9640 1 -- 172.21.15.102:0/1156501557 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f228009ac20 msgr2=0x7f228011c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:32.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.887+0000 7f2285cf9640 1 --2- 172.21.15.102:0/1156501557 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f228009ac20 0x7f228011c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:32.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.887+0000 7f2285cf9640 1 -- 172.21.15.102:0/1156501557 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f228009a340 msgr2=0x7f228011c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:32.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.887+0000 7f2285cf9640 1 --2- 172.21.15.102:0/1156501557 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f228009a340 0x7f228011c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:32.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.887+0000 7f2285cf9640 1 -- 172.21.15.102:0/1156501557 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227c009390 con 0x7f228006b140 2024-09-13T17:43:32.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.887+0000 7f22864fa640 1 --2- 172.21.15.102:0/1156501557 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f228009ac20 0x7f228011c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:32.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.887+0000 7f22854f8640 1 --2- 172.21.15.102:0/1156501557 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f228009a340 0x7f228011c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:32.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.887+0000 7f2285cf9640 1 --2- 172.21.15.102:0/1156501557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f228006b140 0x7f228011f640 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f227c00ea70 tx=0x7f227c0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:32.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.888+0000 7f2275ffb640 1 -- 172.21.15.102:0/1156501557 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c00ca10 con 0x7f228006b140 2024-09-13T17:43:32.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.888+0000 7f2275ffb640 1 -- 172.21.15.102:0/1156501557 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227c00cb70 con 0x7f228006b140 2024-09-13T17:43:32.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.888+0000 7f2287f5d640 1 -- 172.21.15.102:0/1156501557 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2270009390 con 0x7f228006b140 2024-09-13T17:43:32.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.888+0000 7f2275ffb640 1 -- 172.21.15.102:0/1156501557 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c013470 con 0x7f228006b140 2024-09-13T17:43:32.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.888+0000 7f2287f5d640 1 -- 172.21.15.102:0/1156501557 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f228011d9b0 con 0x7f228006b140 2024-09-13T17:43:32.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.889+0000 7f2275ffb640 1 -- 172.21.15.102:0/1156501557 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f227c0137e0 con 0x7f228006b140 2024-09-13T17:43:32.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.889+0000 7f2275ffb640 1 -- 172.21.15.102:0/1156501557 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f227c0423a0 con 0x7f228006b140 2024-09-13T17:43:32.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.889+0000 7f2287f5d640 1 -- 172.21.15.102:0/1156501557 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f228011cbe0 con 0x7f228006b140 2024-09-13T17:43:32.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:32.894+0000 7f2275ffb640 1 -- 172.21.15.102:0/1156501557 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f228011cbe0 con 0x7f228006b140 2024-09-13T17:43:33.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.082+0000 7f2287f5d640 1 -- 172.21.15.102:0/1156501557 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f228011cbe0 con 0x7f228006b140 2024-09-13T17:43:33.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.084+0000 7f2275ffb640 1 -- 172.21.15.102:0/1156501557 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+73559 (secure 0 0 0) 0x7f228011cbe0 con 0x7f228006b140 2024-09-13T17:43:33.086 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:43:33.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.086+0000 7f2287f5d640 1 -- 172.21.15.102:0/1156501557 >> v1:172.21.15.102:6816/34937 conn(0x7f2264031160 legacy=0x7f2264033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.086+0000 7f2287f5d640 1 -- 172.21.15.102:0/1156501557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f228006b140 msgr2=0x7f228011f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.086+0000 7f2287f5d640 1 --2- 172.21.15.102:0/1156501557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f228006b140 0x7f228011f640 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f227c00ea70 tx=0x7f227c0090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.086+0000 7f2287f5d640 1 -- 172.21.15.102:0/1156501557 shutdown_connections 2024-09-13T17:43:33.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.086+0000 7f2287f5d640 1 --2- 172.21.15.102:0/1156501557 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f228009ac20 0x7f228011c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.086+0000 7f2287f5d640 1 --2- 172.21.15.102:0/1156501557 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f228009a340 0x7f228011c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.086+0000 7f2287f5d640 1 --2- 172.21.15.102:0/1156501557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f228006b140 0x7f228011f640 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.086+0000 7f2287f5d640 1 -- 172.21.15.102:0/1156501557 >> 172.21.15.102:0/1156501557 conn(0x7f228009e820 msgr2=0x7f228006cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:33.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.086+0000 7f2287f5d640 1 -- 172.21.15.102:0/1156501557 shutdown_connections 2024-09-13T17:43:33.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.086+0000 7f2287f5d640 1 -- 172.21.15.102:0/1156501557 wait complete. 2024-09-13T17:43:33.098 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6816","nonce":34937}]},"active_addr":"172.21.15.102:6816/34937","active_change":"2024-09-13T17:41:46.407161+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.102:0","nonce":2908116049}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.102:0","nonce":3205190670}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.102:0","nonce":3429257460}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.102:0","nonce":4212379315}]}]} 2024-09-13T17:43:33.100 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-13T17:43:33.100 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T17:43:33.100 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:43:33.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.192+0000 7f328e79d640 1 Processor -- start 2024-09-13T17:43:33.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.192+0000 7f328e79d640 1 -- start start 2024-09-13T17:43:33.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.192+0000 7f328e79d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f328809abb0 0x7f328812c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.192+0000 7f328e79d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f328812c6a0 0x7f328812cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.192+0000 7f328e79d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f328806a530 con 0x7f3288096d80 2024-09-13T17:43:33.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.192+0000 7f328e79d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f328809af80 con 0x7f328812c6a0 2024-09-13T17:43:33.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.192+0000 7f328e79d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f328812cff0 con 0x7f328809abb0 2024-09-13T17:43:33.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.193+0000 7f32877fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f328812c6a0 0x7f328812cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.193+0000 7f3287fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f328809abb0 0x7f328812c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.193+0000 7f328cd3a640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f3288096d80 0x7f3288097200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:56206/0 (socket says 172.21.15.102:56206) 2024-09-13T17:43:33.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.193+0000 7f328cd3a640 1 -- 172.21.15.102:0/239910950 learned_addr learned my addr 172.21.15.102:0/239910950 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:33.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.193+0000 7f32877fe640 1 --2- 172.21.15.102:0/239910950 >> v2:172.21.15.102:3300/0 conn(0x7f328812c6a0 0x7f328812cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57346/0 (socket says 172.21.15.102:57346) 2024-09-13T17:43:33.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.193+0000 7f3287fff640 1 --2- 172.21.15.102:0/239910950 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f328809abb0 0x7f328812c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47102/0 (socket says 172.21.15.102:47102) 2024-09-13T17:43:33.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.193+0000 7f32877fe640 1 -- 172.21.15.102:0/239910950 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f328809abb0 msgr2=0x7f328812c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:33.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.193+0000 7f32877fe640 1 --2- 172.21.15.102:0/239910950 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f328809abb0 0x7f328812c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.193+0000 7f32877fe640 1 -- 172.21.15.102:0/239910950 >> v1:172.21.15.102:6789/0 conn(0x7f3288096d80 legacy=0x7f3288097200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.193+0000 7f3286ffd640 1 -- 172.21.15.102:0/239910950 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3611929563 0 0) 0x7f328806a530 con 0x7f3288096d80 2024-09-13T17:43:33.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.193+0000 7f32877fe640 1 -- 172.21.15.102:0/239910950 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3288098be0 con 0x7f328812c6a0 2024-09-13T17:43:33.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.193+0000 7f3287fff640 1 --2- 172.21.15.102:0/239910950 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f328809abb0 0x7f328812c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:33.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.194+0000 7f32877fe640 1 --2- 172.21.15.102:0/239910950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f328812c6a0 0x7f328812cab0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f327c009880 tx=0x7f327c0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e809102435e4d68d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:33.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.194+0000 7f3286ffd640 1 -- 172.21.15.102:0/239910950 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c02ac50 con 0x7f328812c6a0 2024-09-13T17:43:33.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.194+0000 7f3286ffd640 1 -- 172.21.15.102:0/239910950 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f327c02adb0 con 0x7f328812c6a0 2024-09-13T17:43:33.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.194+0000 7f3286ffd640 1 -- 172.21.15.102:0/239910950 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c032670 con 0x7f328812c6a0 2024-09-13T17:43:33.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.194+0000 7f328e79d640 1 -- 172.21.15.102:0/239910950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f328812c6a0 msgr2=0x7f328812cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.194+0000 7f328e79d640 1 --2- 172.21.15.102:0/239910950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f328812c6a0 0x7f328812cab0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f327c009880 tx=0x7f327c0090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.194+0000 7f328e79d640 1 -- 172.21.15.102:0/239910950 shutdown_connections 2024-09-13T17:43:33.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.194+0000 7f328e79d640 1 --2- 172.21.15.102:0/239910950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f328812c6a0 0x7f328812cab0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.194+0000 7f328e79d640 1 --2- 172.21.15.102:0/239910950 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f328809abb0 0x7f328812c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.194+0000 7f328e79d640 1 -- 172.21.15.102:0/239910950 >> 172.21.15.102:0/239910950 conn(0x7f328809e820 msgr2=0x7f328809ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:33.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.194+0000 7f328e79d640 1 -- 172.21.15.102:0/239910950 shutdown_connections 2024-09-13T17:43:33.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.194+0000 7f328e79d640 1 -- 172.21.15.102:0/239910950 wait complete. 2024-09-13T17:43:33.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.195+0000 7f328e79d640 1 Processor -- start 2024-09-13T17:43:33.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.195+0000 7f328e79d640 1 -- start start 2024-09-13T17:43:33.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.196+0000 7f328e79d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3288096d80 0x7f328811b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.196+0000 7f328e79d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f328809abb0 0x7f3288117c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.196+0000 7f328e79d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f328812c6a0 0x7f3288118150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.196+0000 7f328e79d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3288118f80 con 0x7f328812c6a0 2024-09-13T17:43:33.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.196+0000 7f328e79d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f328809d7d0 con 0x7f328809abb0 2024-09-13T17:43:33.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.196+0000 7f328e79d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32880981a0 con 0x7f3288096d80 2024-09-13T17:43:33.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.196+0000 7f3287fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3288096d80 0x7f328811b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.196+0000 7f328cd3a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f328812c6a0 0x7f3288118150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.196+0000 7f3287fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3288096d80 0x7f328811b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47118/0 (socket says 172.21.15.102:47118) 2024-09-13T17:43:33.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.196+0000 7f3287fff640 1 -- 172.21.15.102:0/1921654045 learned_addr learned my addr 172.21.15.102:0/1921654045 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:33.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.196+0000 7f32877fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f328809abb0 0x7f3288117c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.196+0000 7f3287fff640 1 -- 172.21.15.102:0/1921654045 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f328809abb0 msgr2=0x7f3288117c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.196+0000 7f3287fff640 1 --2- 172.21.15.102:0/1921654045 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f328809abb0 0x7f3288117c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.196+0000 7f3287fff640 1 -- 172.21.15.102:0/1921654045 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f328812c6a0 msgr2=0x7f3288118150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.196+0000 7f3287fff640 1 --2- 172.21.15.102:0/1921654045 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f328812c6a0 0x7f3288118150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.196+0000 7f3287fff640 1 -- 172.21.15.102:0/1921654045 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f327800b630 con 0x7f3288096d80 2024-09-13T17:43:33.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.196+0000 7f328cd3a640 1 --2- 172.21.15.102:0/1921654045 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f328812c6a0 0x7f3288118150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:33.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.196+0000 7f3287fff640 1 --2- 172.21.15.102:0/1921654045 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3288096d80 0x7f328811b390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3278010fc0 tx=0x7f327800b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:33.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.196+0000 7f326ffff640 1 -- 172.21.15.102:0/1921654045 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327800b990 con 0x7f3288096d80 2024-09-13T17:43:33.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.196+0000 7f326ffff640 1 -- 172.21.15.102:0/1921654045 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3278011660 con 0x7f3288096d80 2024-09-13T17:43:33.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.196+0000 7f328e79d640 1 -- 172.21.15.102:0/1921654045 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f327c009390 con 0x7f3288096d80 2024-09-13T17:43:33.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.196+0000 7f326ffff640 1 -- 172.21.15.102:0/1921654045 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3278011a00 con 0x7f3288096d80 2024-09-13T17:43:33.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.197+0000 7f328e79d640 1 -- 172.21.15.102:0/1921654045 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3288118690 con 0x7f3288096d80 2024-09-13T17:43:33.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.197+0000 7f326ffff640 1 -- 172.21.15.102:0/1921654045 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f3278019440 con 0x7f3288096d80 2024-09-13T17:43:33.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.197+0000 7f326ffff640 1 -- 172.21.15.102:0/1921654045 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f3278044e40 con 0x7f3288096d80 2024-09-13T17:43:33.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.197+0000 7f328e79d640 1 -- 172.21.15.102:0/1921654045 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3288098e00 con 0x7f3288096d80 2024-09-13T17:43:33.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.202+0000 7f326ffff640 1 -- 172.21.15.102:0/1921654045 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f32780223b0 con 0x7f3288096d80 2024-09-13T17:43:33.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.354+0000 7f328e79d640 1 -- 172.21.15.102:0/1921654045 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3288064000 con 0x7f3288096d80 2024-09-13T17:43:33.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.355+0000 7f326ffff640 1 -- 172.21.15.102:0/1921654045 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+11660 (secure 0 0 0) 0x7f327800d350 con 0x7f3288096d80 2024-09-13T17:43:33.357 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:43:33.357 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":23,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:42:48.385896+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:04.672265+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:05.583134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.17","mappings":[{"from":3,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]},{"pgid":"4.25","mappings":[{"from":3,"to":2}]},{"pgid":"4.33","mappings":[{"from":3,"to":0}]},{"pgid":"4.3a","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:43:33.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.357+0000 7f328e79d640 1 -- 172.21.15.102:0/1921654045 >> v1:172.21.15.102:6816/34937 conn(0x7f3274031160 legacy=0x7f3274033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.357+0000 7f328e79d640 1 -- 172.21.15.102:0/1921654045 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3288096d80 msgr2=0x7f328811b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.357+0000 7f328e79d640 1 --2- 172.21.15.102:0/1921654045 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3288096d80 0x7f328811b390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3278010fc0 tx=0x7f327800b3b0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.357+0000 7f328e79d640 1 -- 172.21.15.102:0/1921654045 shutdown_connections 2024-09-13T17:43:33.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.357+0000 7f328e79d640 1 --2- 172.21.15.102:0/1921654045 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f328812c6a0 0x7f3288118150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.357+0000 7f328e79d640 1 --2- 172.21.15.102:0/1921654045 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f328809abb0 0x7f3288117c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.357+0000 7f328e79d640 1 --2- 172.21.15.102:0/1921654045 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3288096d80 0x7f328811b390 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.357+0000 7f328e79d640 1 -- 172.21.15.102:0/1921654045 >> 172.21.15.102:0/1921654045 conn(0x7f328809e820 msgr2=0x7f328806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:33.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.357+0000 7f328e79d640 1 -- 172.21.15.102:0/1921654045 shutdown_connections 2024-09-13T17:43:33.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.357+0000 7f328e79d640 1 -- 172.21.15.102:0/1921654045 wait complete. 2024-09-13T17:43:33.370 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T17:43:33.370 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:43:33.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.461+0000 7f5f6be8b640 1 Processor -- start 2024-09-13T17:43:33.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.461+0000 7f5f6be8b640 1 -- start start 2024-09-13T17:43:33.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.461+0000 7f5f6be8b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f64128330 0x7f5f64128700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.461+0000 7f5f6be8b640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5f6412bf50 0x7f5f64133b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.461+0000 7f5f6be8b640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f64097be0 con 0x7f5f64128c40 2024-09-13T17:43:33.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.461+0000 7f5f6be8b640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f64098e70 con 0x7f5f6412bf50 2024-09-13T17:43:33.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.461+0000 7f5f6be8b640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5f64098310 con 0x7f5f64128330 2024-09-13T17:43:33.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.461+0000 7f5f69c27640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f64128330 0x7f5f64128700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.462+0000 7f5f69c27640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f64128330 0x7f5f64128700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47126/0 (socket says 172.21.15.102:47126) 2024-09-13T17:43:33.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.462+0000 7f5f6a428640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5f6412bf50 0x7f5f64133b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.462+0000 7f5f69c27640 1 -- 172.21.15.102:0/3246084022 learned_addr learned my addr 172.21.15.102:0/3246084022 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:33.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.462+0000 7f5f69c27640 1 -- 172.21.15.102:0/3246084022 >> v2:172.21.15.102:3300/0 conn(0x7f5f6412bf50 msgr2=0x7f5f64133b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:33.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.462+0000 7f5f69c27640 1 --2- 172.21.15.102:0/3246084022 >> v2:172.21.15.102:3300/0 conn(0x7f5f6412bf50 0x7f5f64133b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.462+0000 7f5f69c27640 1 -- 172.21.15.102:0/3246084022 >> v1:172.21.15.102:6789/0 conn(0x7f5f64128c40 legacy=0x7f5f6412b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.462+0000 7f5f69c27640 1 -- 172.21.15.102:0/3246084022 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f640985d0 con 0x7f5f64128330 2024-09-13T17:43:33.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.462+0000 7f5f6a428640 1 --2- 172.21.15.102:0/3246084022 >> v2:172.21.15.102:3300/0 conn(0x7f5f6412bf50 0x7f5f64133b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:33.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.462+0000 7f5f69c27640 1 --2- 172.21.15.102:0/3246084022 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f64128330 0x7f5f64128700 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000aac0 tx=0x7f5f6000a360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6946c9a1a7cea93 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:33.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.463+0000 7f5f68c25640 1 -- 172.21.15.102:0/3246084022 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f60030510 con 0x7f5f64128330 2024-09-13T17:43:33.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.463+0000 7f5f68c25640 1 -- 172.21.15.102:0/3246084022 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6002e050 con 0x7f5f64128330 2024-09-13T17:43:33.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.463+0000 7f5f68c25640 1 -- 172.21.15.102:0/3246084022 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f600308e0 con 0x7f5f64128330 2024-09-13T17:43:33.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.463+0000 7f5f6be8b640 1 -- 172.21.15.102:0/3246084022 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f64128330 msgr2=0x7f5f64128700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.463+0000 7f5f6be8b640 1 --2- 172.21.15.102:0/3246084022 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f64128330 0x7f5f64128700 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000aac0 tx=0x7f5f6000a360 comp rx=0 tx=0).stop 2024-09-13T17:43:33.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.463+0000 7f5f6be8b640 1 -- 172.21.15.102:0/3246084022 shutdown_connections 2024-09-13T17:43:33.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.463+0000 7f5f6be8b640 1 --2- 172.21.15.102:0/3246084022 >> v2:172.21.15.102:3300/0 conn(0x7f5f6412bf50 0x7f5f64133b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.463+0000 7f5f6be8b640 1 --2- 172.21.15.102:0/3246084022 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f64128330 0x7f5f64128700 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.463+0000 7f5f6be8b640 1 -- 172.21.15.102:0/3246084022 >> 172.21.15.102:0/3246084022 conn(0x7f5f6409e820 msgr2=0x7f5f6409ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:33.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.463+0000 7f5f6be8b640 1 -- 172.21.15.102:0/3246084022 shutdown_connections 2024-09-13T17:43:33.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.463+0000 7f5f6be8b640 1 -- 172.21.15.102:0/3246084022 wait complete. 2024-09-13T17:43:33.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.464+0000 7f5f6be8b640 1 Processor -- start 2024-09-13T17:43:33.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.464+0000 7f5f6be8b640 1 -- start start 2024-09-13T17:43:33.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.464+0000 7f5f6be8b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f64128330 0x7f5f6406bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.464+0000 7f5f6be8b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f64128c40 0x7f5f6406c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.464+0000 7f5f6be8b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f6412bf50 0x7f5f6406c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.464+0000 7f5f6be8b640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f64125fd0 con 0x7f5f64128c40 2024-09-13T17:43:33.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.464+0000 7f5f6be8b640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6409baa0 con 0x7f5f64128330 2024-09-13T17:43:33.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.464+0000 7f5f6be8b640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6406e930 con 0x7f5f6412bf50 2024-09-13T17:43:33.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.465+0000 7f5f69c27640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f64128330 0x7f5f6406bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.465+0000 7f5f69426640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f64128c40 0x7f5f6406c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.465+0000 7f5f6a428640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f6412bf50 0x7f5f6406c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.465+0000 7f5f69c27640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f64128330 0x7f5f6406bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:39628/0 (socket says 172.21.15.102:39628) 2024-09-13T17:43:33.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.465+0000 7f5f69426640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f64128c40 0x7f5f6406c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57376/0 (socket says 172.21.15.102:57376) 2024-09-13T17:43:33.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.465+0000 7f5f69c27640 1 -- 172.21.15.102:0/56311471 learned_addr learned my addr 172.21.15.102:0/56311471 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:33.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.465+0000 7f5f69c27640 1 -- 172.21.15.102:0/56311471 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f6412bf50 msgr2=0x7f5f6406c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.465+0000 7f5f69c27640 1 --2- 172.21.15.102:0/56311471 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f6412bf50 0x7f5f6406c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.465+0000 7f5f69c27640 1 -- 172.21.15.102:0/56311471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f64128c40 msgr2=0x7f5f6406c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.465+0000 7f5f69c27640 1 --2- 172.21.15.102:0/56311471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f64128c40 0x7f5f6406c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.465+0000 7f5f69c27640 1 -- 172.21.15.102:0/56311471 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f5c009a20 con 0x7f5f64128330 2024-09-13T17:43:33.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.465+0000 7f5f69c27640 1 --2- 172.21.15.102:0/56311471 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f64128330 0x7f5f6406bad0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000aac0 tx=0x7f5f60038000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:33.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.465+0000 7f5f59ffb640 1 -- 172.21.15.102:0/56311471 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f60032890 con 0x7f5f64128330 2024-09-13T17:43:33.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.465+0000 7f5f59ffb640 1 -- 172.21.15.102:0/56311471 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f60032a10 con 0x7f5f64128330 2024-09-13T17:43:33.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.465+0000 7f5f59ffb640 1 -- 172.21.15.102:0/56311471 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f60032dd0 con 0x7f5f64128330 2024-09-13T17:43:33.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.465+0000 7f5f6be8b640 1 -- 172.21.15.102:0/56311471 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f6000a600 con 0x7f5f64128330 2024-09-13T17:43:33.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.465+0000 7f5f6be8b640 1 -- 172.21.15.102:0/56311471 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f641154e0 con 0x7f5f64128330 2024-09-13T17:43:33.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.466+0000 7f5f59ffb640 1 -- 172.21.15.102:0/56311471 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f5f6003c410 con 0x7f5f64128330 2024-09-13T17:43:33.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.466+0000 7f5f59ffb640 1 -- 172.21.15.102:0/56311471 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f5f60062df0 con 0x7f5f64128330 2024-09-13T17:43:33.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.466+0000 7f5f6be8b640 1 -- 172.21.15.102:0/56311471 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f64064000 con 0x7f5f64128330 2024-09-13T17:43:33.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.471+0000 7f5f59ffb640 1 -- 172.21.15.102:0/56311471 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f5f6004a080 con 0x7f5f64128330 2024-09-13T17:43:33.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.625+0000 7f5f6be8b640 1 -- 172.21.15.102:0/56311471 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f64067690 con 0x7f5f64128330 2024-09-13T17:43:33.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.626+0000 7f5f59ffb640 1 -- 172.21.15.102:0/56311471 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+11660 (secure 0 0 0) 0x7f5f60040020 con 0x7f5f64128330 2024-09-13T17:43:33.628 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:43:33.628 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":23,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:42:48.385896+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:04.672265+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:05.583134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.17","mappings":[{"from":3,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]},{"pgid":"4.25","mappings":[{"from":3,"to":2}]},{"pgid":"4.33","mappings":[{"from":3,"to":0}]},{"pgid":"4.3a","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:43:33.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.628+0000 7f5f6be8b640 1 -- 172.21.15.102:0/56311471 >> v1:172.21.15.102:6816/34937 conn(0x7f5f48031160 legacy=0x7f5f48033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.628+0000 7f5f6be8b640 1 -- 172.21.15.102:0/56311471 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f64128330 msgr2=0x7f5f6406bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.628+0000 7f5f6be8b640 1 --2- 172.21.15.102:0/56311471 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f64128330 0x7f5f6406bad0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000aac0 tx=0x7f5f60038000 comp rx=0 tx=0).stop 2024-09-13T17:43:33.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.628+0000 7f5f6be8b640 1 -- 172.21.15.102:0/56311471 shutdown_connections 2024-09-13T17:43:33.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.628+0000 7f5f6be8b640 1 --2- 172.21.15.102:0/56311471 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5f6412bf50 0x7f5f6406c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.628+0000 7f5f6be8b640 1 --2- 172.21.15.102:0/56311471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f64128c40 0x7f5f6406c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.628+0000 7f5f6be8b640 1 --2- 172.21.15.102:0/56311471 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5f64128330 0x7f5f6406bad0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.628+0000 7f5f6be8b640 1 -- 172.21.15.102:0/56311471 >> 172.21.15.102:0/56311471 conn(0x7f5f6409e820 msgr2=0x7f5f6412c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:33.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.628+0000 7f5f6be8b640 1 -- 172.21.15.102:0/56311471 shutdown_connections 2024-09-13T17:43:33.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.628+0000 7f5f6be8b640 1 -- 172.21.15.102:0/56311471 wait complete. 2024-09-13T17:43:33.641 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T17:43:33.641 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T17:43:33.641 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T17:43:33.642 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-13T17:43:33.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.739+0000 7f3cea79d640 1 Processor -- start 2024-09-13T17:43:33.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.739+0000 7fd30ab9d640 1 Processor -- start 2024-09-13T17:43:33.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.739+0000 7f12a5f9d640 1 Processor -- start 2024-09-13T17:43:33.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.739+0000 7fd30ab9d640 1 -- start start 2024-09-13T17:43:33.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.739+0000 7fc7d9ec0640 1 Processor -- start 2024-09-13T17:43:33.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.739+0000 7f12a5f9d640 1 -- start start 2024-09-13T17:43:33.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.739+0000 7f3cea79d640 1 -- start start 2024-09-13T17:43:33.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.740+0000 7fc7d9ec0640 1 -- start start 2024-09-13T17:43:33.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.740+0000 7fd30ab9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd30409abb0 0x7fd30412c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.740+0000 7f12a5f9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f12a009abb0 0x7f12a012c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.740+0000 7fd30ab9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd30412c6a0 0x7fd30412cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.740+0000 7fc7d9ec0640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc7d412c480 0x7fc7d412c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.740+0000 7fd30ab9d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd304096d80 0x7fd304097220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.740+0000 7fc7d9ec0640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc7d412cd90 0x7fc7d409a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.740+0000 7fc7d9ec0640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7d405dd60 con 0x7fc7d409ad50 2024-09-13T17:43:33.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.740+0000 7fc7d9ec0640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d4098e40 con 0x7fc7d412c480 2024-09-13T17:43:33.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.740+0000 7fc7d9ec0640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d412a260 con 0x7fc7d412cd90 2024-09-13T17:43:33.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.740+0000 7fd30ab9d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd30409af80 con 0x7fd304096d80 2024-09-13T17:43:33.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.740+0000 7fd30ab9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd30412cff0 con 0x7fd30409abb0 2024-09-13T17:43:33.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.740+0000 7f3cea79d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ce4068920 0x7f3ce409d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.740+0000 7fc7d3fff640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fc7d409ad50 0x7fc7d4133440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:56236/0 (socket says 172.21.15.102:56236) 2024-09-13T17:43:33.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.740+0000 7fc7d3fff640 1 -- 172.21.15.102:0/3319315317 learned_addr learned my addr 172.21.15.102:0/3319315317 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:33.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.740+0000 7fc7d2ffd640 1 --2- 172.21.15.102:0/3319315317 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc7d412cd90 0x7fc7d409a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.740+0000 7fd30ab9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd30406a530 con 0x7fd30412c6a0 2024-09-13T17:43:33.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.740+0000 7fd308939640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd30409abb0 0x7fd30412c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.740+0000 7f3cea79d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3ce409dc50 0x7f3ce409e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.740+0000 7f3cea79d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ce412f310 0x7f3ce412f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.740+0000 7f3cea79d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ce4068cf0 con 0x7f3ce409dc50 2024-09-13T17:43:33.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.740+0000 7f3cea79d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce40695a0 con 0x7f3ce4068920 2024-09-13T17:43:33.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.740+0000 7f3cea79d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce409f090 con 0x7f3ce412f310 2024-09-13T17:43:33.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.740+0000 7f3ce3fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ce4068920 0x7f3ce409d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.740+0000 7f3ce3fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ce4068920 0x7f3ce409d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:39650/0 (socket says 172.21.15.102:39650) 2024-09-13T17:43:33.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.740+0000 7fc7d37fe640 1 --2- 172.21.15.102:0/3319315317 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc7d412c480 0x7fc7d412c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.740+0000 7f3ce3fff640 1 -- 172.21.15.102:0/150854430 learned_addr learned my addr 172.21.15.102:0/150854430 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:33.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.740+0000 7fd308939640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd30409abb0 0x7fd30412c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:39660/0 (socket says 172.21.15.102:39660) 2024-09-13T17:43:33.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.740+0000 7fd308939640 1 -- 172.21.15.102:0/2422257020 learned_addr learned my addr 172.21.15.102:0/2422257020 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:33.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.740+0000 7fd30913a640 1 --2- 172.21.15.102:0/2422257020 >> v2:172.21.15.102:3300/0 conn(0x7fd304096d80 0x7fd304097220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.740+0000 7fd303fff640 1 --2- 172.21.15.102:0/2422257020 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd30412c6a0 0x7fd30412cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.741+0000 7fd308939640 1 -- 172.21.15.102:0/2422257020 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd30412c6a0 msgr2=0x7fd30412cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:33.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.741+0000 7fd308939640 1 --2- 172.21.15.102:0/2422257020 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd30412c6a0 0x7fd30412cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.741+0000 7fd308939640 1 -- 172.21.15.102:0/2422257020 >> v2:172.21.15.102:3300/0 conn(0x7fd304096d80 msgr2=0x7fd304097220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:33.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.741+0000 7fd308939640 1 --2- 172.21.15.102:0/2422257020 >> v2:172.21.15.102:3300/0 conn(0x7fd304096d80 0x7fd304097220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.741+0000 7fd308939640 1 -- 172.21.15.102:0/2422257020 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd304098be0 con 0x7fd30409abb0 2024-09-13T17:43:33.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.740+0000 7f3ce37fe640 1 --2- 172.21.15.102:0/150854430 >> v2:172.21.15.102:3300/0 conn(0x7f3ce409dc50 0x7f3ce409e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.741+0000 7f3ce3fff640 1 -- 172.21.15.102:0/150854430 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ce412f310 msgr2=0x7f3ce412f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:33.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.741+0000 7f3ce3fff640 1 --2- 172.21.15.102:0/150854430 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ce412f310 0x7f3ce412f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.741+0000 7f3ce3fff640 1 -- 172.21.15.102:0/150854430 >> v2:172.21.15.102:3300/0 conn(0x7f3ce409dc50 msgr2=0x7f3ce409e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:33.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.741+0000 7f3ce3fff640 1 --2- 172.21.15.102:0/150854430 >> v2:172.21.15.102:3300/0 conn(0x7f3ce409dc50 0x7f3ce409e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.741+0000 7f3ce3fff640 1 -- 172.21.15.102:0/150854430 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce409cce0 con 0x7f3ce4068920 2024-09-13T17:43:33.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.740+0000 7f129f7fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f12a009abb0 0x7f12a012c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.740+0000 7f129f7fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f12a009abb0 0x7f12a012c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:39640/0 (socket says 172.21.15.102:39640) 2024-09-13T17:43:33.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.741+0000 7f129f7fe640 1 -- 172.21.15.102:0/3535058869 learned_addr learned my addr 172.21.15.102:0/3535058869 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:33.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.741+0000 7fc7d2ffd640 1 -- 172.21.15.102:0/3319315317 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc7d412c480 msgr2=0x7fc7d412c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:33.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.741+0000 7fc7d2ffd640 1 --2- 172.21.15.102:0/3319315317 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc7d412c480 0x7fc7d412c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.741+0000 7fc7d2ffd640 1 -- 172.21.15.102:0/3319315317 >> v1:172.21.15.102:6789/0 conn(0x7fc7d409ad50 legacy=0x7fc7d4133440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.741+0000 7fc7d2ffd640 1 -- 172.21.15.102:0/3319315317 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d4097d00 con 0x7fc7d412cd90 2024-09-13T17:43:33.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.741+0000 7fc7d37fe640 1 --2- 172.21.15.102:0/3319315317 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc7d412c480 0x7fc7d412c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:33.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.741+0000 7f129effd640 1 --1- 172.21.15.102:0/3535058869 >> v1:172.21.15.102:6789/0 conn(0x7f12a012c6a0 0x7f12a0096d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:56230/0 (socket says 172.21.15.102:56230) 2024-09-13T17:43:33.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.741+0000 7f12a5f9d640 1 --2- 172.21.15.102:0/3535058869 >> v2:172.21.15.102:3300/0 conn(0x7f12a012cab0 0x7f12a0097490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.741+0000 7f12a5f9d640 1 -- 172.21.15.102:0/3535058869 --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12a006a530 con 0x7f12a012c6a0 2024-09-13T17:43:33.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.741+0000 7f12a5f9d640 1 -- 172.21.15.102:0/3535058869 --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12a009af80 con 0x7f12a012cab0 2024-09-13T17:43:33.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.741+0000 7f12a5f9d640 1 -- 172.21.15.102:0/3535058869 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a0098be0 con 0x7f12a009abb0 2024-09-13T17:43:33.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.741+0000 7f129e7fc640 1 -- 172.21.15.102:0/3535058869 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 772957365 0 0) 0x7f12a006a530 con 0x7f12a012c6a0 2024-09-13T17:43:33.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.741+0000 7f129e7fc640 1 -- 172.21.15.102:0/3535058869 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12880034e0 con 0x7f12a012c6a0 2024-09-13T17:43:33.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.741+0000 7f129ffff640 1 --2- 172.21.15.102:0/3535058869 >> v2:172.21.15.102:3300/0 conn(0x7f12a012cab0 0x7f12a0097490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.741+0000 7f129e7fc640 1 -- 172.21.15.102:0/3535058869 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 884799909 0 0) 0x7f12880034e0 con 0x7f12a012c6a0 2024-09-13T17:43:33.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7f129e7fc640 1 -- 172.21.15.102:0/3535058869 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f12a009abb0 msgr2=0x7f12a012c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:33.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7f129e7fc640 1 --2- 172.21.15.102:0/3535058869 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f12a009abb0 0x7f12a012c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7f129e7fc640 1 -- 172.21.15.102:0/3535058869 >> v2:172.21.15.102:3300/0 conn(0x7f12a012cab0 msgr2=0x7f12a0097490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:33.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7f129e7fc640 1 --2- 172.21.15.102:0/3535058869 >> v2:172.21.15.102:3300/0 conn(0x7f12a012cab0 0x7f12a0097490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7f129e7fc640 1 -- 172.21.15.102:0/3535058869 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a0098930 con 0x7f12a012c6a0 2024-09-13T17:43:33.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7f129e7fc640 1 -- 172.21.15.102:0/3535058869 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1399394525 0 0) 0x7f1294001470 con 0x7f12a012c6a0 2024-09-13T17:43:33.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7f129f7fe640 1 --2- 172.21.15.102:0/3535058869 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f12a009abb0 0x7f12a012c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:33.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7f129e7fc640 1 -- 172.21.15.102:0/3535058869 >> v1:172.21.15.102:6789/0 conn(0x7f12a012c6a0 legacy=0x7f12a0096d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7f129e7fc640 1 --2- 172.21.15.102:0/3535058869 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f12a009abb0 0x7f12880039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7f129e7fc640 1 --2- 172.21.15.102:0/3535058869 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1288003f10 0x7f1288004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7f129e7fc640 1 --2- 172.21.15.102:0/3535058869 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1288004cc0 0x7f1288005130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7f129e7fc640 1 -- 172.21.15.102:0/3535058869 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a0098be0 con 0x7f1288004cc0 2024-09-13T17:43:33.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7f129e7fc640 1 -- 172.21.15.102:0/3535058869 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f128800dc10 con 0x7f12a009abb0 2024-09-13T17:43:33.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7f129e7fc640 1 -- 172.21.15.102:0/3535058869 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f128800ddd0 con 0x7f1288003f10 2024-09-13T17:43:33.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7f129f7fe640 1 --2- 172.21.15.102:0/3535058869 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f12a009abb0 0x7f12880039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7fc7d2ffd640 1 --2- 172.21.15.102:0/3319315317 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc7d412cd90 0x7fc7d409a810 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8009880 tx=0x7fc7c80090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28fbcff1b4335da5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:33.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7f3ce3fff640 1 --2- 172.21.15.102:0/150854430 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ce4068920 0x7f3ce409d710 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4009910 tx=0x7f3cd40091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e066064992b9260f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:33.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7f129effd640 1 --2- 172.21.15.102:0/3535058869 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1288003f10 0x7f1288004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7f129f7fe640 1 -- 172.21.15.102:0/3535058869 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1288003f10 msgr2=0x7f1288004780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:33.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7f129f7fe640 1 --2- 172.21.15.102:0/3535058869 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1288003f10 0x7f1288004780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7f129f7fe640 1 -- 172.21.15.102:0/3535058869 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1288004cc0 msgr2=0x7f1288005130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:43:33.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7f129f7fe640 1 --2- 172.21.15.102:0/3535058869 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1288004cc0 0x7f1288005130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7f129f7fe640 1 -- 172.21.15.102:0/3535058869 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1290009450 con 0x7f12a009abb0 2024-09-13T17:43:33.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7fd308939640 1 --2- 172.21.15.102:0/2422257020 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd30409abb0 0x7fd30412c160 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd2f800bbf0 tx=0x7fd2f800b460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=849b407346ddcdce server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:33.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7f129effd640 1 --2- 172.21.15.102:0/3535058869 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1288003f10 0x7f1288004780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:43:33.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7fc7d27fc640 1 -- 172.21.15.102:0/3319315317 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c802ac60 con 0x7fc7d412cd90 2024-09-13T17:43:33.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7fc7d27fc640 1 -- 172.21.15.102:0/3319315317 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c802adc0 con 0x7fc7d412cd90 2024-09-13T17:43:33.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7fc7d27fc640 1 -- 172.21.15.102:0/3319315317 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c802ac60 con 0x7fc7d412cd90 2024-09-13T17:43:33.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7f3ce2ffd640 1 -- 172.21.15.102:0/150854430 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd402ad20 con 0x7f3ce4068920 2024-09-13T17:43:33.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7f3ce2ffd640 1 -- 172.21.15.102:0/150854430 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd402ae80 con 0x7f3ce4068920 2024-09-13T17:43:33.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7f3ce2ffd640 1 -- 172.21.15.102:0/150854430 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4032670 con 0x7f3ce4068920 2024-09-13T17:43:33.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7f129ffff640 1 --2- 172.21.15.102:0/3535058869 >> v2:172.21.15.102:3300/0 conn(0x7f12a012cab0 0x7f12a0097490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:43:33.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7fd3037fe640 1 -- 172.21.15.102:0/2422257020 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f8031510 con 0x7fd30409abb0 2024-09-13T17:43:33.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7fd3037fe640 1 -- 172.21.15.102:0/2422257020 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f802f050 con 0x7fd30409abb0 2024-09-13T17:43:33.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.742+0000 7fd3037fe640 1 -- 172.21.15.102:0/2422257020 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f80318a0 con 0x7fd30409abb0 2024-09-13T17:43:33.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.743+0000 7fc7d9ec0640 1 -- 172.21.15.102:0/3319315317 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc7d412cd90 msgr2=0x7fc7d409a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.743+0000 7fc7d9ec0640 1 --2- 172.21.15.102:0/3319315317 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc7d412cd90 0x7fc7d409a810 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8009880 tx=0x7fc7c80090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.743+0000 7f3cea79d640 1 -- 172.21.15.102:0/150854430 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ce4068920 msgr2=0x7f3ce409d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.743+0000 7fc7d9ec0640 1 -- 172.21.15.102:0/3319315317 shutdown_connections 2024-09-13T17:43:33.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.743+0000 7f3cea79d640 1 --2- 172.21.15.102:0/150854430 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ce4068920 0x7f3ce409d710 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4009910 tx=0x7f3cd40091d0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.743+0000 7fc7d9ec0640 1 --2- 172.21.15.102:0/3319315317 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc7d412cd90 0x7fc7d409a810 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.743+0000 7fc7d9ec0640 1 --2- 172.21.15.102:0/3319315317 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc7d412c480 0x7fc7d412c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.743+0000 7fc7d9ec0640 1 -- 172.21.15.102:0/3319315317 >> 172.21.15.102:0/3319315317 conn(0x7fc7d409f8b0 msgr2=0x7fc7d409b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:33.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.743+0000 7fc7d9ec0640 1 -- 172.21.15.102:0/3319315317 shutdown_connections 2024-09-13T17:43:33.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.743+0000 7fc7d9ec0640 1 -- 172.21.15.102:0/3319315317 wait complete. 2024-09-13T17:43:33.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.743+0000 7f3cea79d640 1 -- 172.21.15.102:0/150854430 shutdown_connections 2024-09-13T17:43:33.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.743+0000 7f3cea79d640 1 --2- 172.21.15.102:0/150854430 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ce412f310 0x7f3ce412f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.743+0000 7f3cea79d640 1 --2- 172.21.15.102:0/150854430 >> v2:172.21.15.102:3300/0 conn(0x7f3ce409dc50 0x7f3ce409e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.743+0000 7f3cea79d640 1 --2- 172.21.15.102:0/150854430 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ce4068920 0x7f3ce409d710 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.743+0000 7f3cea79d640 1 -- 172.21.15.102:0/150854430 >> 172.21.15.102:0/150854430 conn(0x7f3ce406f320 msgr2=0x7f3ce40a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:33.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.743+0000 7f3cea79d640 1 -- 172.21.15.102:0/150854430 shutdown_connections 2024-09-13T17:43:33.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.743+0000 7fd30ab9d640 1 -- 172.21.15.102:0/2422257020 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd30409abb0 msgr2=0x7fd30412c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.743+0000 7fd30ab9d640 1 --2- 172.21.15.102:0/2422257020 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd30409abb0 0x7fd30412c160 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd2f800bbf0 tx=0x7fd2f800b460 comp rx=0 tx=0).stop 2024-09-13T17:43:33.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.743+0000 7f3cea79d640 1 -- 172.21.15.102:0/150854430 wait complete. 2024-09-13T17:43:33.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.743+0000 7f129f7fe640 1 --2- 172.21.15.102:0/3535058869 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f12a009abb0 0x7f12880039d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1290006f30 tx=0x7f12900091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8329db8655a6f16 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:33.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.743+0000 7fd30ab9d640 1 -- 172.21.15.102:0/2422257020 shutdown_connections 2024-09-13T17:43:33.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.743+0000 7fd30ab9d640 1 --2- 172.21.15.102:0/2422257020 >> v2:172.21.15.102:3300/0 conn(0x7fd304096d80 0x7fd304097220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.743+0000 7fd30ab9d640 1 --2- 172.21.15.102:0/2422257020 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd30412c6a0 0x7fd30412cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.743+0000 7fd30ab9d640 1 --2- 172.21.15.102:0/2422257020 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd30409abb0 0x7fd30412c160 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.743+0000 7fd30ab9d640 1 -- 172.21.15.102:0/2422257020 >> 172.21.15.102:0/2422257020 conn(0x7fd30409e820 msgr2=0x7fd30409ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:33.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.743+0000 7fd30ab9d640 1 -- 172.21.15.102:0/2422257020 shutdown_connections 2024-09-13T17:43:33.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.743+0000 7fd30ab9d640 1 -- 172.21.15.102:0/2422257020 wait complete. 2024-09-13T17:43:33.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.743+0000 7f129e7fc640 1 -- 172.21.15.102:0/3535058869 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1290032410 con 0x7f12a009abb0 2024-09-13T17:43:33.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.744+0000 7f129e7fc640 1 -- 172.21.15.102:0/3535058869 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1290032570 con 0x7f12a009abb0 2024-09-13T17:43:33.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.744+0000 7f129e7fc640 1 -- 172.21.15.102:0/3535058869 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1290032910 con 0x7f12a009abb0 2024-09-13T17:43:33.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.744+0000 7fc7d9ec0640 1 Processor -- start 2024-09-13T17:43:33.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.744+0000 7fc7d9ec0640 1 -- start start 2024-09-13T17:43:33.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.744+0000 7f3cea79d640 1 Processor -- start 2024-09-13T17:43:33.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.744+0000 7fd30ab9d640 1 Processor -- start 2024-09-13T17:43:33.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.744+0000 7f12a5f9d640 1 -- 172.21.15.102:0/3535058869 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f12a009abb0 msgr2=0x7f12880039d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.744+0000 7f12a5f9d640 1 --2- 172.21.15.102:0/3535058869 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f12a009abb0 0x7f12880039d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1290006f30 tx=0x7f12900091d0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.744+0000 7f129ffff640 1 -- 172.21.15.102:0/3535058869 reap_dead start 2024-09-13T17:43:33.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.744+0000 7f3cea79d640 1 -- start start 2024-09-13T17:43:33.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.745+0000 7f12a5f9d640 1 -- 172.21.15.102:0/3535058869 shutdown_connections 2024-09-13T17:43:33.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.745+0000 7f12a5f9d640 1 -- 172.21.15.102:0/3535058869 >> 172.21.15.102:0/3535058869 conn(0x7f12a009e820 msgr2=0x7f12a009ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:33.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.745+0000 7fd30ab9d640 1 -- start start 2024-09-13T17:43:33.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.745+0000 7f12a5f9d640 1 -- 172.21.15.102:0/3535058869 shutdown_connections 2024-09-13T17:43:33.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.745+0000 7f12a5f9d640 1 -- 172.21.15.102:0/3535058869 wait complete. 2024-09-13T17:43:33.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.745+0000 7fc7d9ec0640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc7d409ad50 0x7fc7d413a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.745+0000 7fc7d9ec0640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc7d412c480 0x7fc7d413aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.745+0000 7fc7d9ec0640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7d412cd90 0x7fc7d4123ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.745+0000 7fc7d9ec0640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d4129f60 con 0x7fc7d412cd90 2024-09-13T17:43:33.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.745+0000 7fc7d9ec0640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d4127910 con 0x7fc7d412c480 2024-09-13T17:43:33.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.745+0000 7fc7d9ec0640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d4125180 con 0x7fc7d409ad50 2024-09-13T17:43:33.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.745+0000 7fc7d2ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc7d412c480 0x7fc7d413aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.745+0000 7fc7d37fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc7d409ad50 0x7fc7d413a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.745+0000 7fc7d37fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc7d409ad50 0x7fc7d413a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47168/0 (socket says 172.21.15.102:47168) 2024-09-13T17:43:33.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.745+0000 7fc7d37fe640 1 -- 172.21.15.102:0/2880603024 learned_addr learned my addr 172.21.15.102:0/2880603024 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:33.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.745+0000 7fc7d3fff640 1 --2- 172.21.15.102:0/2880603024 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7d412cd90 0x7fc7d4123ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.745+0000 7f3cea79d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ce4068920 0x7f3ce4121e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.745+0000 7fd30ab9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd304096d80 0x7fd30413a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.745+0000 7fd30ab9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd30409abb0 0x7fd30413ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.745+0000 7f3cea79d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ce409dc50 0x7f3ce411dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.745+0000 7f3cea79d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ce412f310 0x7f3ce411e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.745+0000 7f3cea79d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce411ff20 con 0x7f3ce412f310 2024-09-13T17:43:33.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.745+0000 7f3cea79d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce4121640 con 0x7f3ce409dc50 2024-09-13T17:43:33.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.745+0000 7f3cea79d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce409ff40 con 0x7f3ce4068920 2024-09-13T17:43:33.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.746+0000 7f3ce3fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ce4068920 0x7f3ce4121e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.745+0000 7fd30ab9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd30412c6a0 0x7fd3041153c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.745+0000 7fd30913a640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd30412c6a0 0x7fd3041153c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.746+0000 7fd30913a640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd30412c6a0 0x7fd3041153c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47188/0 (socket says 172.21.15.102:47188) 2024-09-13T17:43:33.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.745+0000 7fd308939640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd304096d80 0x7fd30413a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.745+0000 7fc7d37fe640 1 -- 172.21.15.102:0/2880603024 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc7d412c480 msgr2=0x7fc7d413aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.746+0000 7fd308939640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd304096d80 0x7fd30413a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:39696/0 (socket says 172.21.15.102:39696) 2024-09-13T17:43:33.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.746+0000 7fd30913a640 1 -- 172.21.15.102:0/1556114973 learned_addr learned my addr 172.21.15.102:0/1556114973 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:33.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.746+0000 7fd303fff640 1 --2- 172.21.15.102:0/1556114973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd30409abb0 0x7fd30413ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.746+0000 7fd30ab9d640 1 -- 172.21.15.102:0/1556114973 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30409a1f0 con 0x7fd30409abb0 2024-09-13T17:43:33.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.746+0000 7fd30ab9d640 1 -- 172.21.15.102:0/1556114973 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3040981a0 con 0x7fd304096d80 2024-09-13T17:43:33.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.746+0000 7fd30ab9d640 1 -- 172.21.15.102:0/1556114973 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd304118da0 con 0x7fd30412c6a0 2024-09-13T17:43:33.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.746+0000 7f3ce3fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ce4068920 0x7f3ce4121e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47182/0 (socket says 172.21.15.102:47182) 2024-09-13T17:43:33.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.746+0000 7f3ce3fff640 1 -- 172.21.15.102:0/2972654654 learned_addr learned my addr 172.21.15.102:0/2972654654 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:33.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.746+0000 7f3ce37fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ce409dc50 0x7f3ce411dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.745+0000 7fc7d37fe640 1 --2- 172.21.15.102:0/2880603024 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc7d412c480 0x7fc7d413aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.745+0000 7fc7d37fe640 1 -- 172.21.15.102:0/2880603024 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7d412cd90 msgr2=0x7fc7d4123ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.746+0000 7f3ce8d3a640 1 --2- 172.21.15.102:0/2972654654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ce412f310 0x7f3ce411e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.746+0000 7f12a5f9d640 1 Processor -- start 2024-09-13T17:43:33.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.745+0000 7fc7d37fe640 1 --2- 172.21.15.102:0/2880603024 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7d412cd90 0x7fc7d4123ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.745+0000 7fc7d37fe640 1 -- 172.21.15.102:0/2880603024 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c400b630 con 0x7fc7d409ad50 2024-09-13T17:43:33.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.746+0000 7fc7d37fe640 1 --2- 172.21.15.102:0/2880603024 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc7d409ad50 0x7fc7d413a9a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4010f70 tx=0x7fc7c400b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:33.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.746+0000 7fc7bb7fe640 1 -- 172.21.15.102:0/2880603024 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c400eea0 con 0x7fc7d409ad50 2024-09-13T17:43:33.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.746+0000 7fc7bb7fe640 1 -- 172.21.15.102:0/2880603024 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c4011660 con 0x7fc7d409ad50 2024-09-13T17:43:33.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.746+0000 7fc7bb7fe640 1 -- 172.21.15.102:0/2880603024 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c40119e0 con 0x7fc7d409ad50 2024-09-13T17:43:33.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.746+0000 7f12a5f9d640 1 -- start start 2024-09-13T17:43:33.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.746+0000 7fd30913a640 1 -- 172.21.15.102:0/1556114973 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd304096d80 msgr2=0x7fd30413a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.746+0000 7fd30913a640 1 --2- 172.21.15.102:0/1556114973 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd304096d80 0x7fd30413a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.746+0000 7fd30913a640 1 -- 172.21.15.102:0/1556114973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd30409abb0 msgr2=0x7fd30413ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.746+0000 7fd30913a640 1 --2- 172.21.15.102:0/1556114973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd30409abb0 0x7fd30413ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.746+0000 7f3ce37fe640 1 -- 172.21.15.102:0/2972654654 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ce4068920 msgr2=0x7f3ce4121e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.746+0000 7f3ce37fe640 1 --2- 172.21.15.102:0/2972654654 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3ce4068920 0x7f3ce4121e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.746+0000 7f3ce37fe640 1 -- 172.21.15.102:0/2972654654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ce412f310 msgr2=0x7f3ce411e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.746+0000 7f3ce37fe640 1 --2- 172.21.15.102:0/2972654654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ce412f310 0x7f3ce411e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.746+0000 7f3ce37fe640 1 -- 172.21.15.102:0/2972654654 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd4009450 con 0x7f3ce409dc50 2024-09-13T17:43:33.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.746+0000 7f3ce8d3a640 1 --2- 172.21.15.102:0/2972654654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ce412f310 0x7f3ce411e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:43:33.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.746+0000 7fd30913a640 1 -- 172.21.15.102:0/1556114973 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f800b700 con 0x7fd30412c6a0 2024-09-13T17:43:33.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.746+0000 7fd308939640 1 --2- 172.21.15.102:0/1556114973 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd304096d80 0x7fd30413a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:43:33.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.747+0000 7f3ce37fe640 1 --2- 172.21.15.102:0/2972654654 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ce409dc50 0x7f3ce411dee0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800e9c0 tx=0x7f3cd80090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:33.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.746+0000 7fc7d9ec0640 1 -- 172.21.15.102:0/2880603024 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c8009390 con 0x7fc7d409ad50 2024-09-13T17:43:33.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.746+0000 7fc7d9ec0640 1 -- 172.21.15.102:0/2880603024 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d413b420 con 0x7fc7d409ad50 2024-09-13T17:43:33.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.747+0000 7fd30913a640 1 --2- 172.21.15.102:0/1556114973 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd30412c6a0 0x7fd3041153c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc009df0 tx=0x7fd2fc00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:33.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.747+0000 7fc7bb7fe640 1 -- 172.21.15.102:0/2880603024 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fc7c4011d50 con 0x7fc7d409ad50 2024-09-13T17:43:33.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.747+0000 7f3ccbfff640 1 -- 172.21.15.102:0/2972654654 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd800c940 con 0x7f3ce409dc50 2024-09-13T17:43:33.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.747+0000 7fd300ff9640 1 -- 172.21.15.102:0/1556114973 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc00cd40 con 0x7fd30412c6a0 2024-09-13T17:43:33.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.747+0000 7fd300ff9640 1 -- 172.21.15.102:0/1556114973 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2fc00f040 con 0x7fd30412c6a0 2024-09-13T17:43:33.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.747+0000 7fd300ff9640 1 -- 172.21.15.102:0/1556114973 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc0146d0 con 0x7fd30412c6a0 2024-09-13T17:43:33.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.747+0000 7f3ccbfff640 1 -- 172.21.15.102:0/2972654654 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd800caa0 con 0x7f3ce409dc50 2024-09-13T17:43:33.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.747+0000 7fd30ab9d640 1 -- 172.21.15.102:0/1556114973 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd30412f530 con 0x7fd30412c6a0 2024-09-13T17:43:33.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.747+0000 7fd30ab9d640 1 -- 172.21.15.102:0/1556114973 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd304116f00 con 0x7fd30412c6a0 2024-09-13T17:43:33.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.747+0000 7fd303fff640 1 --2- 172.21.15.102:0/1556114973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd30409abb0 0x7fd30413ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:43:33.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.747+0000 7f12a5f9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f12a011b3a0 0x7f12a0117bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.747+0000 7f12a5f9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12a0118130 0x7f12a0118500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.747+0000 7f12a5f9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f12a013a410 0x7f12a013a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.747+0000 7f12a5f9d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a011b770 con 0x7f12a0118130 2024-09-13T17:43:33.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.747+0000 7f12a5f9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a009c720 con 0x7f12a011b3a0 2024-09-13T17:43:33.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.747+0000 7f12a5f9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12a00981a0 con 0x7f12a013a410 2024-09-13T17:43:33.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.747+0000 7f129f7fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f12a011b3a0 0x7f12a0117bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.747+0000 7f129f7fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f12a011b3a0 0x7f12a0117bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:39702/0 (socket says 172.21.15.102:39702) 2024-09-13T17:43:33.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.747+0000 7f129f7fe640 1 -- 172.21.15.102:0/3531133531 learned_addr learned my addr 172.21.15.102:0/3531133531 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:33.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.747+0000 7fc7d9ec0640 1 -- 172.21.15.102:0/2880603024 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc7b4000fd0 con 0x7fc7d409ad50 2024-09-13T17:43:33.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.747+0000 7fc7bb7fe640 1 -- 172.21.15.102:0/2880603024 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7fc7c4042a90 con 0x7fc7d409ad50 2024-09-13T17:43:33.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.747+0000 7fc7bb7fe640 1 -- 172.21.15.102:0/2880603024 --> v1:172.21.15.102:6812/35352 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc7c0042190 con 0x7fc7c003e490 2024-09-13T17:43:33.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.748+0000 7fc7bb7fe640 1 -- 172.21.15.102:0/2880603024 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7c40223e0 con 0x7fc7d409ad50 2024-09-13T17:43:33.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.748+0000 7f129effd640 1 --2- 172.21.15.102:0/3531133531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12a0118130 0x7f12a0118500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.748+0000 7f129ffff640 1 --2- 172.21.15.102:0/3531133531 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f12a013a410 0x7f12a013a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.748+0000 7fd300ff9640 1 -- 172.21.15.102:0/1556114973 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fd2fc014a80 con 0x7fd30412c6a0 2024-09-13T17:43:33.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.748+0000 7f129ffff640 1 -- 172.21.15.102:0/3531133531 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f12a011b3a0 msgr2=0x7f12a0117bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.748+0000 7f129ffff640 1 --2- 172.21.15.102:0/3531133531 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f12a011b3a0 0x7f12a0117bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.748+0000 7f129ffff640 1 -- 172.21.15.102:0/3531133531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12a0118130 msgr2=0x7f12a0118500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.748+0000 7f129ffff640 1 --2- 172.21.15.102:0/3531133531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12a0118130 0x7f12a0118500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.748+0000 7f129ffff640 1 -- 172.21.15.102:0/3531133531 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a011a8e0 con 0x7f12a013a410 2024-09-13T17:43:33.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.748+0000 7f129effd640 1 --2- 172.21.15.102:0/3531133531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12a0118130 0x7f12a0118500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:33.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.748+0000 7fd30ab9d640 1 -- 172.21.15.102:0/1556114973 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd304098e00 con 0x7fd30412c6a0 2024-09-13T17:43:33.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.747+0000 7f3cea79d640 1 -- 172.21.15.102:0/2972654654 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce4120df0 con 0x7f3ce409dc50 2024-09-13T17:43:33.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.747+0000 7f3cea79d640 1 -- 172.21.15.102:0/2972654654 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce411e960 con 0x7f3ce409dc50 2024-09-13T17:43:33.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.748+0000 7f3ccbfff640 1 -- 172.21.15.102:0/2972654654 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd800ce40 con 0x7f3ce409dc50 2024-09-13T17:43:33.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.749+0000 7f129ffff640 1 --2- 172.21.15.102:0/3531133531 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f12a013a410 0x7f12a013a7e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f12a006a6c0 tx=0x7f1298009870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:33.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.749+0000 7f3ccbfff640 1 -- 172.21.15.102:0/2972654654 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f3cd8013680 con 0x7f3ce409dc50 2024-09-13T17:43:33.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.749+0000 7fd300ff9640 1 -- 172.21.15.102:0/1556114973 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7fd2fc042ae0 con 0x7fd30412c6a0 2024-09-13T17:43:33.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.749+0000 7fd300ff9640 1 -- 172.21.15.102:0/1556114973 --> v1:172.21.15.102:6800/35327 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd2e803dd80 con 0x7fd2e803a080 2024-09-13T17:43:33.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.749+0000 7fd300ff9640 1 -- 172.21.15.102:0/1556114973 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2fc044ea0 con 0x7fd30412c6a0 2024-09-13T17:43:33.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.749+0000 7f3ccbfff640 1 -- 172.21.15.102:0/2972654654 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f3cd8042930 con 0x7f3ce409dc50 2024-09-13T17:43:33.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.749+0000 7f129f7fe640 1 --2- 172.21.15.102:0/3531133531 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f12a011b3a0 0x7f12a0117bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:33.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.749+0000 7f12877fe640 1 -- 172.21.15.102:0/3531133531 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1298015440 con 0x7f12a013a410 2024-09-13T17:43:33.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.749+0000 7f12877fe640 1 -- 172.21.15.102:0/3531133531 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129800f040 con 0x7f12a013a410 2024-09-13T17:43:33.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.749+0000 7f12a5f9d640 1 -- 172.21.15.102:0/3531133531 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a0117970 con 0x7f12a013a410 2024-09-13T17:43:33.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.749+0000 7fc7bb7fe640 1 -- 172.21.15.102:0/2880603024 <== osd.3 v1:172.21.15.102:6812/35352 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7fc7c0042190 con 0x7fc7c003e490 2024-09-13T17:43:33.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.749+0000 7f12877fe640 1 -- 172.21.15.102:0/3531133531 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12980157d0 con 0x7f12a013a410 2024-09-13T17:43:33.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.749+0000 7f12a5f9d640 1 -- 172.21.15.102:0/3531133531 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a013ad20 con 0x7f12a013a410 2024-09-13T17:43:33.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.749+0000 7f3cc9ffb640 1 -- 172.21.15.102:0/2972654654 --> v1:172.21.15.102:6804/35328 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3ce411a8a0 con 0x7f3ce40632a0 2024-09-13T17:43:33.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.750+0000 7f12a5f9d640 1 -- 172.21.15.102:0/3531133531 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f12a0064000 con 0x7f12a013a410 2024-09-13T17:43:33.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.750+0000 7f12877fe640 1 -- 172.21.15.102:0/3531133531 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f129801e3b0 con 0x7f12a013a410 2024-09-13T17:43:33.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.750+0000 7fd300ff9640 1 -- 172.21.15.102:0/1556114973 <== osd.0 v1:172.21.15.102:6800/35327 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7fd2e803dd80 con 0x7fd2e803a080 2024-09-13T17:43:33.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.751+0000 7f12877fe640 1 -- 172.21.15.102:0/3531133531 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f12980418f0 con 0x7f12a013a410 2024-09-13T17:43:33.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.751+0000 7f12877fe640 1 -- 172.21.15.102:0/3531133531 --> v1:172.21.15.102:6808/35351 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f128c03ddc0 con 0x7f128c03a0c0 2024-09-13T17:43:33.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.751+0000 7f12877fe640 1 -- 172.21.15.102:0/3531133531 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1298014560 con 0x7f12a013a410 2024-09-13T17:43:33.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.751+0000 7f3ccbfff640 1 -- 172.21.15.102:0/2972654654 <== osd.1 v1:172.21.15.102:6804/35328 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f3ce411a8a0 con 0x7f3ce40632a0 2024-09-13T17:43:33.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.752+0000 7f12877fe640 1 -- 172.21.15.102:0/3531133531 <== osd.2 v1:172.21.15.102:6808/35351 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f128c03ddc0 con 0x7f128c03a0c0 2024-09-13T17:43:33.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.763+0000 7fc7d9ec0640 1 -- 172.21.15.102:0/2880603024 --> v1:172.21.15.102:6812/35352 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc7b4002c00 con 0x7fc7c003e490 2024-09-13T17:43:33.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.763+0000 7fc7bb7fe640 1 -- 172.21.15.102:0/2880603024 <== osd.3 v1:172.21.15.102:6812/35352 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 2189488832) 0x7fc7b4002c00 con 0x7fc7c003e490 2024-09-13T17:43:33.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.763+0000 7fc7d9ec0640 1 -- 172.21.15.102:0/2880603024 >> v1:172.21.15.102:6812/35352 conn(0x7fc7c003e490 legacy=0x7fc7c0040920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.763+0000 7fc7d9ec0640 1 -- 172.21.15.102:0/2880603024 >> v1:172.21.15.102:6816/34937 conn(0x7fc7c00355a0 legacy=0x7fc7c0037a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.763+0000 7fc7d9ec0640 1 -- 172.21.15.102:0/2880603024 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc7d409ad50 msgr2=0x7fc7d413a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.763+0000 7fc7d9ec0640 1 --2- 172.21.15.102:0/2880603024 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc7d409ad50 0x7fc7d413a9a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4010f70 tx=0x7fc7c400b3b0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.763+0000 7fc7d3fff640 1 -- 172.21.15.102:0/2880603024 reap_dead start 2024-09-13T17:43:33.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.764+0000 7fc7d9ec0640 1 -- 172.21.15.102:0/2880603024 shutdown_connections 2024-09-13T17:43:33.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.764+0000 7fc7d9ec0640 1 -- 172.21.15.102:0/2880603024 >> 172.21.15.102:0/2880603024 conn(0x7fc7d409f8b0 msgr2=0x7fc7d4099800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:33.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.764+0000 7fc7d9ec0640 1 -- 172.21.15.102:0/2880603024 shutdown_connections 2024-09-13T17:43:33.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.764+0000 7fc7d9ec0640 1 -- 172.21.15.102:0/2880603024 wait complete. 2024-09-13T17:43:33.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.765+0000 7f12a5f9d640 1 -- 172.21.15.102:0/3531133531 --> v1:172.21.15.102:6808/35351 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f12a013b1b0 con 0x7f128c03a0c0 2024-09-13T17:43:33.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.765+0000 7f12877fe640 1 -- 172.21.15.102:0/3531133531 <== osd.2 v1:172.21.15.102:6808/35351 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 2189488832) 0x7f12a013b1b0 con 0x7f128c03a0c0 2024-09-13T17:43:33.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.766+0000 7f12a5f9d640 1 -- 172.21.15.102:0/3531133531 >> v1:172.21.15.102:6808/35351 conn(0x7f128c03a0c0 legacy=0x7f128c03c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.766+0000 7f12a5f9d640 1 -- 172.21.15.102:0/3531133531 >> v1:172.21.15.102:6816/34937 conn(0x7f128c031160 legacy=0x7f128c033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.766+0000 7f12a5f9d640 1 -- 172.21.15.102:0/3531133531 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f12a013a410 msgr2=0x7f12a013a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.766+0000 7f12a5f9d640 1 --2- 172.21.15.102:0/3531133531 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f12a013a410 0x7f12a013a7e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f12a006a6c0 tx=0x7f1298009870 comp rx=0 tx=0).stop 2024-09-13T17:43:33.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.766+0000 7f129ffff640 1 -- 172.21.15.102:0/3531133531 reap_dead start 2024-09-13T17:43:33.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.766+0000 7f12a5f9d640 1 -- 172.21.15.102:0/3531133531 shutdown_connections 2024-09-13T17:43:33.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.766+0000 7f12a5f9d640 1 -- 172.21.15.102:0/3531133531 >> 172.21.15.102:0/3531133531 conn(0x7f12a009e820 msgr2=0x7f12a0099a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:33.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.766+0000 7f12a5f9d640 1 -- 172.21.15.102:0/3531133531 shutdown_connections 2024-09-13T17:43:33.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.766+0000 7f12a5f9d640 1 -- 172.21.15.102:0/3531133531 wait complete. 2024-09-13T17:43:33.791 INFO:teuthology.orchestra.run.smithi102.stdout:38654705686 2024-09-13T17:43:33.791 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T17:43:33.792 INFO:teuthology.orchestra.run.smithi102.stdout:38654705686 2024-09-13T17:43:33.792 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T17:43:33.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.779+0000 7fd30ab9d640 1 -- 172.21.15.102:0/1556114973 --> v1:172.21.15.102:6800/35327 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd304064000 con 0x7fd2e803a080 2024-09-13T17:43:33.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.779+0000 7f3cc9ffb640 1 -- 172.21.15.102:0/2972654654 --> v1:172.21.15.102:6804/35328 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3ce4063d80 con 0x7f3ce40632a0 2024-09-13T17:43:33.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.779+0000 7fd300ff9640 1 -- 172.21.15.102:0/1556114973 <== osd.0 v1:172.21.15.102:6800/35327 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 1894478275) 0x7fd304064000 con 0x7fd2e803a080 2024-09-13T17:43:33.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.779+0000 7f3ccbfff640 1 -- 172.21.15.102:0/2972654654 <== osd.1 v1:172.21.15.102:6804/35328 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 3700674276) 0x7f3ce4063d80 con 0x7f3ce40632a0 2024-09-13T17:43:33.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.780+0000 7fd30ab9d640 1 -- 172.21.15.102:0/1556114973 >> v1:172.21.15.102:6800/35327 conn(0x7fd2e803a080 legacy=0x7fd2e803c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.780+0000 7f3cc9ffb640 1 -- 172.21.15.102:0/2972654654 >> v1:172.21.15.102:6804/35328 conn(0x7f3ce40632a0 legacy=0x7f3ce4063670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.780+0000 7fd30ab9d640 1 -- 172.21.15.102:0/1556114973 >> v1:172.21.15.102:6816/34937 conn(0x7fd2e8031190 legacy=0x7fd2e8033640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.780+0000 7fd30ab9d640 1 -- 172.21.15.102:0/1556114973 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd30412c6a0 msgr2=0x7fd3041153c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.780+0000 7fd30ab9d640 1 --2- 172.21.15.102:0/1556114973 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd30412c6a0 0x7fd3041153c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc009df0 tx=0x7fd2fc00b040 comp rx=0 tx=0).stop 2024-09-13T17:43:33.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.780+0000 7f3cc9ffb640 1 -- 172.21.15.102:0/2972654654 >> v1:172.21.15.102:6816/34937 conn(0x7f3cd0031230 legacy=0x7f3cd00336e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.780+0000 7fd30913a640 1 -- 172.21.15.102:0/1556114973 reap_dead start 2024-09-13T17:43:33.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.780+0000 7f3cc9ffb640 1 -- 172.21.15.102:0/2972654654 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ce409dc50 msgr2=0x7f3ce411dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.780+0000 7f3cc9ffb640 1 --2- 172.21.15.102:0/2972654654 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3ce409dc50 0x7f3ce411dee0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800e9c0 tx=0x7f3cd80090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.780+0000 7f3ce8d3a640 1 -- 172.21.15.102:0/2972654654 reap_dead start 2024-09-13T17:43:33.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.780+0000 7f3cc9ffb640 1 -- 172.21.15.102:0/2972654654 shutdown_connections 2024-09-13T17:43:33.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.780+0000 7f3cc9ffb640 1 -- 172.21.15.102:0/2972654654 >> 172.21.15.102:0/2972654654 conn(0x7f3ce406f320 msgr2=0x7f3ce409aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:33.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.780+0000 7fd30ab9d640 1 -- 172.21.15.102:0/1556114973 shutdown_connections 2024-09-13T17:43:33.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.780+0000 7fd30ab9d640 1 -- 172.21.15.102:0/1556114973 >> 172.21.15.102:0/1556114973 conn(0x7fd30409e820 msgr2=0x7fd30409c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:33.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.780+0000 7f3cc9ffb640 1 -- 172.21.15.102:0/2972654654 shutdown_connections 2024-09-13T17:43:33.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.780+0000 7fd30ab9d640 1 -- 172.21.15.102:0/1556114973 shutdown_connections 2024-09-13T17:43:33.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.780+0000 7f3cc9ffb640 1 -- 172.21.15.102:0/2972654654 wait complete. 2024-09-13T17:43:33.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.780+0000 7fd30ab9d640 1 -- 172.21.15.102:0/1556114973 wait complete. 2024-09-13T17:43:33.798 INFO:teuthology.orchestra.run.smithi102.stdout:38654705689 2024-09-13T17:43:33.798 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T17:43:33.799 INFO:teuthology.orchestra.run.smithi102.stdout:38654705687 2024-09-13T17:43:33.800 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T17:43:33.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.893+0000 7febc0b5d640 1 Processor -- start 2024-09-13T17:43:33.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.893+0000 7febc0b5d640 1 -- start start 2024-09-13T17:43:33.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.894+0000 7febc0b5d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7febbc068920 0x7febbc09d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.894+0000 7febc0b5d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7febbc09dc50 0x7febbc09e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.894+0000 7febc0b5d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7febbc12f310 0x7febbc12f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.894+0000 7febc0b5d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febbc068cf0 con 0x7febbc09dc50 2024-09-13T17:43:33.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.894+0000 7febc0b5d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febbc0695a0 con 0x7febbc12f310 2024-09-13T17:43:33.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.894+0000 7febc0b5d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7febbc09f090 con 0x7febbc068920 2024-09-13T17:43:33.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.894+0000 7febba59c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7febbc068920 0x7febbc09d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.894+0000 7febbad9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7febbc12f310 0x7febbc12f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.894+0000 7febba59c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7febbc068920 0x7febbc09d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47210/0 (socket says 172.21.15.102:47210) 2024-09-13T17:43:33.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.894+0000 7febba59c640 1 -- 172.21.15.102:0/1772991904 learned_addr learned my addr 172.21.15.102:0/1772991904 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:33.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.894+0000 7febb9d9b640 1 --2- 172.21.15.102:0/1772991904 >> v2:172.21.15.102:3300/0 conn(0x7febbc09dc50 0x7febbc09e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.894+0000 7febbad9d640 1 -- 172.21.15.102:0/1772991904 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7febbc068920 msgr2=0x7febbc09d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:33.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.894+0000 7febbad9d640 1 --2- 172.21.15.102:0/1772991904 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7febbc068920 0x7febbc09d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.894+0000 7febbad9d640 1 -- 172.21.15.102:0/1772991904 >> v2:172.21.15.102:3300/0 conn(0x7febbc09dc50 msgr2=0x7febbc09e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:33.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.894+0000 7febbad9d640 1 --2- 172.21.15.102:0/1772991904 >> v2:172.21.15.102:3300/0 conn(0x7febbc09dc50 0x7febbc09e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.894+0000 7febbad9d640 1 -- 172.21.15.102:0/1772991904 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febbc09cce0 con 0x7febbc12f310 2024-09-13T17:43:33.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.895+0000 7febbad9d640 1 --2- 172.21.15.102:0/1772991904 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7febbc12f310 0x7febbc12f790 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7febb4009e60 tx=0x7febb4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81eba57123c8f679 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:33.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.896+0000 7febb959a640 1 -- 172.21.15.102:0/1772991904 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb4007550 con 0x7febbc12f310 2024-09-13T17:43:33.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.896+0000 7febb959a640 1 -- 172.21.15.102:0/1772991904 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb402d050 con 0x7febbc12f310 2024-09-13T17:43:33.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.896+0000 7febb959a640 1 -- 172.21.15.102:0/1772991904 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb4007920 con 0x7febbc12f310 2024-09-13T17:43:33.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.896+0000 7febc0b5d640 1 -- 172.21.15.102:0/1772991904 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7febbc12f310 msgr2=0x7febbc12f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.896+0000 7febc0b5d640 1 --2- 172.21.15.102:0/1772991904 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7febbc12f310 0x7febbc12f790 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7febb4009e60 tx=0x7febb4009780 comp rx=0 tx=0).stop 2024-09-13T17:43:33.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.896+0000 7febc0b5d640 1 -- 172.21.15.102:0/1772991904 shutdown_connections 2024-09-13T17:43:33.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.896+0000 7febc0b5d640 1 --2- 172.21.15.102:0/1772991904 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7febbc12f310 0x7febbc12f790 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.897+0000 7febc0b5d640 1 --2- 172.21.15.102:0/1772991904 >> v2:172.21.15.102:3300/0 conn(0x7febbc09dc50 0x7febbc09e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.897+0000 7febc0b5d640 1 --2- 172.21.15.102:0/1772991904 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7febbc068920 0x7febbc09d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.897+0000 7febc0b5d640 1 -- 172.21.15.102:0/1772991904 >> 172.21.15.102:0/1772991904 conn(0x7febbc06f320 msgr2=0x7febbc0a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:33.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.897+0000 7febc0b5d640 1 -- 172.21.15.102:0/1772991904 shutdown_connections 2024-09-13T17:43:33.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.897+0000 7febc0b5d640 1 -- 172.21.15.102:0/1772991904 wait complete. 2024-09-13T17:43:33.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.898+0000 7febc0b5d640 1 Processor -- start 2024-09-13T17:43:33.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.898+0000 7febc0b5d640 1 -- start start 2024-09-13T17:43:33.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.898+0000 7febc0b5d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7febbc068920 0x7febbc13ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.898+0000 7febc0b5d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7febbc09dc50 0x7febbc13f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.899+0000 7febc0b5d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febbc12f310 0x7febbc11a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.899+0000 7febc0b5d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febbc0a2c50 con 0x7febbc12f310 2024-09-13T17:43:33.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.899+0000 7febbad9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febbc12f310 0x7febbc11a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.899+0000 7febb9d9b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7febbc09dc50 0x7febbc13f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.899+0000 7febba59c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7febbc068920 0x7febbc13ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.899+0000 7febb9d9b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7febbc09dc50 0x7febbc13f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47214/0 (socket says 172.21.15.102:47214) 2024-09-13T17:43:33.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.899+0000 7febb9d9b640 1 -- 172.21.15.102:0/3300737322 learned_addr learned my addr 172.21.15.102:0/3300737322 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:33.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.899+0000 7febc0b5d640 1 -- 172.21.15.102:0/3300737322 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febbc11bea0 con 0x7febbc068920 2024-09-13T17:43:33.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.899+0000 7febc0b5d640 1 -- 172.21.15.102:0/3300737322 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7febbc11d810 con 0x7febbc09dc50 2024-09-13T17:43:33.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.899+0000 7febba59c640 1 -- 172.21.15.102:0/3300737322 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7febbc09dc50 msgr2=0x7febbc13f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.899+0000 7febba59c640 1 --2- 172.21.15.102:0/3300737322 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7febbc09dc50 0x7febbc13f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.899+0000 7febba59c640 1 -- 172.21.15.102:0/3300737322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febbc12f310 msgr2=0x7febbc11a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.899+0000 7febba59c640 1 --2- 172.21.15.102:0/3300737322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febbc12f310 0x7febbc11a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.899+0000 7febba59c640 1 -- 172.21.15.102:0/3300737322 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb4009a20 con 0x7febbc068920 2024-09-13T17:43:33.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.899+0000 7febba59c640 1 --2- 172.21.15.102:0/3300737322 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7febbc068920 0x7febbc13ec40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7febac00e900 tx=0x7febac0091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:33.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.900+0000 7febaa7fc640 1 -- 172.21.15.102:0/3300737322 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac00ee90 con 0x7febbc068920 2024-09-13T17:43:33.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.900+0000 7febaa7fc640 1 -- 172.21.15.102:0/3300737322 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febac00c940 con 0x7febbc068920 2024-09-13T17:43:33.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.900+0000 7febc0b5d640 1 -- 172.21.15.102:0/3300737322 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febbc11d540 con 0x7febbc068920 2024-09-13T17:43:33.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.900+0000 7febaa7fc640 1 -- 172.21.15.102:0/3300737322 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac00cce0 con 0x7febbc068920 2024-09-13T17:43:33.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.900+0000 7febc0b5d640 1 -- 172.21.15.102:0/3300737322 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febbc12efe0 con 0x7febbc068920 2024-09-13T17:43:33.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.901+0000 7febc0b5d640 1 -- 172.21.15.102:0/3300737322 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febbc064000 con 0x7febbc068920 2024-09-13T17:43:33.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.901+0000 7febaa7fc640 1 -- 172.21.15.102:0/3300737322 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7febac013680 con 0x7febbc068920 2024-09-13T17:43:33.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.901+0000 7febaa7fc640 1 -- 172.21.15.102:0/3300737322 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7febac042bb0 con 0x7febbc068920 2024-09-13T17:43:33.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.909+0000 7febaa7fc640 1 -- 172.21.15.102:0/3300737322 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7febac013cb0 con 0x7febbc068920 2024-09-13T17:43:33.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.940+0000 7fda8e2c9640 1 Processor -- start 2024-09-13T17:43:33.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.940+0000 7fda8e2c9640 1 -- start start 2024-09-13T17:43:33.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.941+0000 7fda8e2c9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fda8806d720 0x7fda8806daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.941+0000 7fda8e2c9640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fda8809da70 0x7fda8813bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.941+0000 7fda8e2c9640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda88069050 con 0x7fda8806e030 2024-09-13T17:43:33.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.941+0000 7fda8e2c9640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda8806b4e0 con 0x7fda8809da70 2024-09-13T17:43:33.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.941+0000 7fda8e2c9640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fda8809cef0 con 0x7fda8806d720 2024-09-13T17:43:33.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.941+0000 7fda8c866640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fda8809da70 0x7fda8813bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.941+0000 7fda8c866640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fda8809da70 0x7fda8813bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57466/0 (socket says 172.21.15.102:57466) 2024-09-13T17:43:33.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.941+0000 7fda8c866640 1 -- 172.21.15.102:0/580572162 learned_addr learned my addr 172.21.15.102:0/580572162 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:33.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.941+0000 7fda87fff640 1 --2- 172.21.15.102:0/580572162 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fda8806d720 0x7fda8806daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.941+0000 7fda86ffd640 1 -- 172.21.15.102:0/580572162 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 359950136 0 0) 0x7fda88069050 con 0x7fda8806e030 2024-09-13T17:43:33.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.941+0000 7fda86ffd640 1 -- 172.21.15.102:0/580572162 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda700034e0 con 0x7fda8806e030 2024-09-13T17:43:33.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.942+0000 7fda87fff640 1 -- 172.21.15.102:0/580572162 >> v2:172.21.15.102:3300/0 conn(0x7fda8809da70 msgr2=0x7fda8813bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:33.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.942+0000 7fda87fff640 1 --2- 172.21.15.102:0/580572162 >> v2:172.21.15.102:3300/0 conn(0x7fda8809da70 0x7fda8813bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.942+0000 7fda87fff640 1 -- 172.21.15.102:0/580572162 >> v1:172.21.15.102:6789/0 conn(0x7fda8806e030 legacy=0x7fda8809d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.942+0000 7fda87fff640 1 -- 172.21.15.102:0/580572162 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda8806e400 con 0x7fda8806d720 2024-09-13T17:43:33.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.942+0000 7fda8c866640 1 --2- 172.21.15.102:0/580572162 >> v2:172.21.15.102:3300/0 conn(0x7fda8809da70 0x7fda8813bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:33.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.942+0000 7fda87fff640 1 --2- 172.21.15.102:0/580572162 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fda8806d720 0x7fda8806daf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fda78009850 tx=0x7fda780090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bdb32a53849bfc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:33.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.943+0000 7fda86ffd640 1 -- 172.21.15.102:0/580572162 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7802ac60 con 0x7fda8806d720 2024-09-13T17:43:33.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.943+0000 7fd32e95d640 1 Processor -- start 2024-09-13T17:43:33.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.943+0000 7fd32e95d640 1 -- start start 2024-09-13T17:43:33.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.943+0000 7fda86ffd640 1 -- 172.21.15.102:0/580572162 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda7802adc0 con 0x7fda8806d720 2024-09-13T17:43:33.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.943+0000 7fda86ffd640 1 -- 172.21.15.102:0/580572162 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda78032650 con 0x7fda8806d720 2024-09-13T17:43:33.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.943+0000 7fd32e95d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd328068990 0x7fd32809d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.943+0000 7fd32e95d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd32809db40 0x7fd32809df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.943+0000 7fd32e95d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd32812f410 0x7fd32812f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.944+0000 7fda8e2c9640 1 -- 172.21.15.102:0/580572162 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fda8806d720 msgr2=0x7fda8806daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.944+0000 7fda8e2c9640 1 --2- 172.21.15.102:0/580572162 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fda8806d720 0x7fda8806daf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fda78009850 tx=0x7fda780090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.944+0000 7fda8e2c9640 1 -- 172.21.15.102:0/580572162 shutdown_connections 2024-09-13T17:43:33.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.944+0000 7fda8e2c9640 1 --2- 172.21.15.102:0/580572162 >> v2:172.21.15.102:3300/0 conn(0x7fda8809da70 0x7fda8813bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.944+0000 7fd327fff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd328068990 0x7fd32809d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.944+0000 7fd32e95d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd328068d60 con 0x7fd328068990 2024-09-13T17:43:33.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.944+0000 7fd327fff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd328068990 0x7fd32809d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57474/0 (socket says 172.21.15.102:57474) 2024-09-13T17:43:33.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.944+0000 7fd327fff640 1 -- 172.21.15.102:0/194750534 learned_addr learned my addr 172.21.15.102:0/194750534 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:33.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.944+0000 7fd32cefa640 1 --2- 172.21.15.102:0/194750534 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd32812f410 0x7fd32812f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.944+0000 7fda8e2c9640 1 --2- 172.21.15.102:0/580572162 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fda8806d720 0x7fda8806daf0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.944+0000 7fda8e2c9640 1 -- 172.21.15.102:0/580572162 >> 172.21.15.102:0/580572162 conn(0x7fda8806f320 msgr2=0x7fda880a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:33.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.944+0000 7fda8e2c9640 1 -- 172.21.15.102:0/580572162 shutdown_connections 2024-09-13T17:43:33.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.944+0000 7fd32e95d640 1 -- 172.21.15.102:0/194750534 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd328097590 con 0x7fd32809db40 2024-09-13T17:43:33.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.944+0000 7fd32e95d640 1 -- 172.21.15.102:0/194750534 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd3280692d0 con 0x7fd32812f410 2024-09-13T17:43:33.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.944+0000 7fda8e2c9640 1 -- 172.21.15.102:0/580572162 wait complete. 2024-09-13T17:43:33.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.944+0000 7fd32cefa640 1 -- 172.21.15.102:0/194750534 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd32809db40 msgr2=0x7fd32809df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:33.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.944+0000 7fd3277fe640 1 --2- 172.21.15.102:0/194750534 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd32809db40 0x7fd32809df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.944+0000 7fd32cefa640 1 --2- 172.21.15.102:0/194750534 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd32809db40 0x7fd32809df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.944+0000 7fd32cefa640 1 -- 172.21.15.102:0/194750534 >> v2:172.21.15.102:3300/0 conn(0x7fd328068990 msgr2=0x7fd32809d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:33.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.944+0000 7fd32cefa640 1 --2- 172.21.15.102:0/194750534 >> v2:172.21.15.102:3300/0 conn(0x7fd328068990 0x7fd32809d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.944+0000 7fd32cefa640 1 -- 172.21.15.102:0/194750534 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd32809f380 con 0x7fd32812f410 2024-09-13T17:43:33.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.944+0000 7fd327fff640 1 --2- 172.21.15.102:0/194750534 >> v2:172.21.15.102:3300/0 conn(0x7fd328068990 0x7fd32809d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:33.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.944+0000 7fd3277fe640 1 --2- 172.21.15.102:0/194750534 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd32809db40 0x7fd32809df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:43:33.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.945+0000 7fda8e2c9640 1 Processor -- start 2024-09-13T17:43:33.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.945+0000 7fda8e2c9640 1 -- start start 2024-09-13T17:43:33.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.945+0000 7fd32cefa640 1 --2- 172.21.15.102:0/194750534 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd32812f410 0x7fd32812f890 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd320009e60 tx=0x7fd320009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29e2e7e3b8412086 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:33.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.945+0000 7fd326ffd640 1 -- 172.21.15.102:0/194750534 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd320007550 con 0x7fd32812f410 2024-09-13T17:43:33.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.945+0000 7fd326ffd640 1 -- 172.21.15.102:0/194750534 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd32002d050 con 0x7fd32812f410 2024-09-13T17:43:33.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.945+0000 7fd326ffd640 1 -- 172.21.15.102:0/194750534 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd320007920 con 0x7fd32812f410 2024-09-13T17:43:33.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.946+0000 7fda8e2c9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fda8806d720 0x7fda88142e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.946+0000 7fda8e2c9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda8806e030 0x7fda881433d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.946+0000 7fda8e2c9640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fda8809da70 0x7fda88122dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.946+0000 7fda87fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fda8806d720 0x7fda88142e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.946+0000 7fda87fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fda8806d720 0x7fda88142e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47250/0 (socket says 172.21.15.102:47250) 2024-09-13T17:43:33.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.946+0000 7fda877fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda8806e030 0x7fda881433d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.946+0000 7fda877fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda8806e030 0x7fda881433d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57488/0 (socket says 172.21.15.102:57488) 2024-09-13T17:43:33.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.946+0000 7fda8c866640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fda8809da70 0x7fda88122dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.946+0000 7fda87fff640 1 -- 172.21.15.102:0/1653286906 learned_addr learned my addr 172.21.15.102:0/1653286906 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:33.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.946+0000 7fda8e2c9640 1 -- 172.21.15.102:0/1653286906 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda88137c50 con 0x7fda8806e030 2024-09-13T17:43:33.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.946+0000 7fda8e2c9640 1 -- 172.21.15.102:0/1653286906 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda880a1980 con 0x7fda8809da70 2024-09-13T17:43:33.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.946+0000 7fda8e2c9640 1 -- 172.21.15.102:0/1653286906 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fda8809e470 con 0x7fda8806d720 2024-09-13T17:43:33.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.946+0000 7fd32e95d640 1 -- 172.21.15.102:0/194750534 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd32812f410 msgr2=0x7fd32812f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.946+0000 7fd32e95d640 1 --2- 172.21.15.102:0/194750534 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd32812f410 0x7fd32812f890 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd320009e60 tx=0x7fd320009780 comp rx=0 tx=0).stop 2024-09-13T17:43:33.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.946+0000 7fda8c866640 1 -- 172.21.15.102:0/1653286906 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fda8806d720 msgr2=0x7fda88142e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.946+0000 7fd32e95d640 1 -- 172.21.15.102:0/194750534 shutdown_connections 2024-09-13T17:43:33.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.946+0000 7fd32e95d640 1 --2- 172.21.15.102:0/194750534 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd32812f410 0x7fd32812f890 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.946+0000 7fd32e95d640 1 --2- 172.21.15.102:0/194750534 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd32809db40 0x7fd32809df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.946+0000 7fd32e95d640 1 --2- 172.21.15.102:0/194750534 >> v2:172.21.15.102:3300/0 conn(0x7fd328068990 0x7fd32809d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.946+0000 7fd32e95d640 1 -- 172.21.15.102:0/194750534 >> 172.21.15.102:0/194750534 conn(0x7fd32806f480 msgr2=0x7fd3280a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:33.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.947+0000 7fd32e95d640 1 -- 172.21.15.102:0/194750534 shutdown_connections 2024-09-13T17:43:33.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.947+0000 7fd32e95d640 1 -- 172.21.15.102:0/194750534 wait complete. 2024-09-13T17:43:33.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.947+0000 7fda8c866640 1 --2- 172.21.15.102:0/1653286906 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fda8806d720 0x7fda88142e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.947+0000 7fda8c866640 1 -- 172.21.15.102:0/1653286906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda8806e030 msgr2=0x7fda881433d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.947+0000 7fda8c866640 1 --2- 172.21.15.102:0/1653286906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda8806e030 0x7fda881433d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.947+0000 7fda8c866640 1 -- 172.21.15.102:0/1653286906 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda80009ae0 con 0x7fda8809da70 2024-09-13T17:43:33.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.947+0000 7fda877fe640 1 --2- 172.21.15.102:0/1653286906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda8806e030 0x7fda881433d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:33.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.947+0000 7fda87fff640 1 --2- 172.21.15.102:0/1653286906 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fda8806d720 0x7fda88142e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:33.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.947+0000 7fc113fff640 1 Processor -- start 2024-09-13T17:43:33.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.947+0000 7fc113fff640 1 -- start start 2024-09-13T17:43:33.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.947+0000 7fda8c866640 1 --2- 172.21.15.102:0/1653286906 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fda8809da70 0x7fda88122dc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fda80009f40 tx=0x7fda8000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:33.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.947+0000 7fda6ffff640 1 -- 172.21.15.102:0/1653286906 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda80018070 con 0x7fda8809da70 2024-09-13T17:43:33.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.947+0000 7fda6ffff640 1 -- 172.21.15.102:0/1653286906 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda8000f040 con 0x7fda8809da70 2024-09-13T17:43:33.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.947+0000 7fda6ffff640 1 -- 172.21.15.102:0/1653286906 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda800092e0 con 0x7fda8809da70 2024-09-13T17:43:33.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.947+0000 7fda8e2c9640 1 -- 172.21.15.102:0/1653286906 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda78009390 con 0x7fda8809da70 2024-09-13T17:43:33.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.947+0000 7fda8e2c9640 1 -- 172.21.15.102:0/1653286906 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda88124850 con 0x7fda8809da70 2024-09-13T17:43:33.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.948+0000 7fd32e95d640 1 Processor -- start 2024-09-13T17:43:33.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.948+0000 7fc113fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc11409bd20 0x7fc11409c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.948+0000 7fc1127fc640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fc114097650 0x7fc114068800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:56240/0 (socket says 172.21.15.102:56240) 2024-09-13T17:43:33.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.948+0000 7fc1127fc640 1 -- 172.21.15.102:0/3502934512 learned_addr learned my addr 172.21.15.102:0/3502934512 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:33.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.948+0000 7fc112ffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc11409bd20 0x7fc11409c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.948+0000 7fc113fff640 1 --2- 172.21.15.102:0/3502934512 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc11409c630 0x7fc114068f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.948+0000 7fc1137fe640 1 --2- 172.21.15.102:0/3502934512 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc11409c630 0x7fc114068f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.948+0000 7fc113fff640 1 -- 172.21.15.102:0/3502934512 --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc11409cce0 con 0x7fc114097650 2024-09-13T17:43:33.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.948+0000 7fda8e2c9640 1 -- 172.21.15.102:0/1653286906 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda88124290 con 0x7fda8809da70 2024-09-13T17:43:33.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.948+0000 7fc113fff640 1 -- 172.21.15.102:0/3502934512 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1140973f0 con 0x7fc11409c630 2024-09-13T17:43:33.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.948+0000 7fc113fff640 1 -- 172.21.15.102:0/3502934512 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc114097ae0 con 0x7fc11409bd20 2024-09-13T17:43:33.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.948+0000 7fc111ffb640 1 -- 172.21.15.102:0/3502934512 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2270119284 0 0) 0x7fc11409cce0 con 0x7fc114097650 2024-09-13T17:43:33.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.948+0000 7fc111ffb640 1 -- 172.21.15.102:0/3502934512 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0fc0034e0 con 0x7fc114097650 2024-09-13T17:43:33.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.948+0000 7fda6ffff640 1 -- 172.21.15.102:0/1653286906 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fda80003020 con 0x7fda8809da70 2024-09-13T17:43:33.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.949+0000 7fda6ffff640 1 -- 172.21.15.102:0/1653286906 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7fda80042110 con 0x7fda8809da70 2024-09-13T17:43:33.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.949+0000 7fc1137fe640 1 -- 172.21.15.102:0/3502934512 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc11409bd20 msgr2=0x7fc11409c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:33.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.949+0000 7fc1137fe640 1 --2- 172.21.15.102:0/3502934512 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc11409bd20 0x7fc11409c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.949+0000 7fc1137fe640 1 -- 172.21.15.102:0/3502934512 >> v1:172.21.15.102:6789/0 conn(0x7fc114097650 legacy=0x7fc114068800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.949+0000 7fc1137fe640 1 -- 172.21.15.102:0/3502934512 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11409dc50 con 0x7fc11409c630 2024-09-13T17:43:33.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.948+0000 7fd32e95d640 1 -- start start 2024-09-13T17:43:33.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.949+0000 7fd32e95d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd328068990 0x7fd32806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.949+0000 7fd32e95d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd32809db40 0x7fd32806aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.949+0000 7fd327fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd328068990 0x7fd32806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.950+0000 7fd32e95d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd32812f410 0x7fd32806b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.950+0000 7fd32e95d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32809e730 con 0x7fd32812f410 2024-09-13T17:43:33.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.950+0000 7fd32e95d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd32806d880 con 0x7fd32809db40 2024-09-13T17:43:33.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.950+0000 7fd32e95d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd32806db40 con 0x7fd328068990 2024-09-13T17:43:33.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.950+0000 7fc1137fe640 1 --2- 172.21.15.102:0/3502934512 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc11409c630 0x7fc114068f10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc10c009e90 tx=0x7fc10c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e25d956b529cfbb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:33.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.950+0000 7fd327fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd328068990 0x7fd32806e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47266/0 (socket says 172.21.15.102:47266) 2024-09-13T17:43:33.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.950+0000 7fd327fff640 1 -- 172.21.15.102:0/408616483 learned_addr learned my addr 172.21.15.102:0/408616483 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:33.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.950+0000 7fd32cefa640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd32812f410 0x7fd32806b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.950+0000 7fd3277fe640 1 --2- 172.21.15.102:0/408616483 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd32809db40 0x7fd32806aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.950+0000 7fd327fff640 1 -- 172.21.15.102:0/408616483 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd32809db40 msgr2=0x7fd32806aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.950+0000 7fd327fff640 1 --2- 172.21.15.102:0/408616483 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd32809db40 0x7fd32806aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.950+0000 7fd327fff640 1 -- 172.21.15.102:0/408616483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd32812f410 msgr2=0x7fd32806b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.950+0000 7fd327fff640 1 --2- 172.21.15.102:0/408616483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd32812f410 0x7fd32806b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.950+0000 7fd327fff640 1 -- 172.21.15.102:0/408616483 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd318009450 con 0x7fd328068990 2024-09-13T17:43:33.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.950+0000 7fd32cefa640 1 --2- 172.21.15.102:0/408616483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd32812f410 0x7fd32806b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:33.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.950+0000 7fd327fff640 1 --2- 172.21.15.102:0/408616483 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd328068990 0x7fd32806e320 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd31800ec80 tx=0x7fd3180091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:33.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.950+0000 7fd30ffff640 1 -- 172.21.15.102:0/408616483 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31800eec0 con 0x7fd328068990 2024-09-13T17:43:33.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.950+0000 7fd30ffff640 1 -- 172.21.15.102:0/408616483 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd31800cbc0 con 0x7fd328068990 2024-09-13T17:43:33.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.950+0000 7fd32e95d640 1 -- 172.21.15.102:0/408616483 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd320009a20 con 0x7fd328068990 2024-09-13T17:43:33.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.951+0000 7fd30ffff640 1 -- 172.21.15.102:0/408616483 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd318013470 con 0x7fd328068990 2024-09-13T17:43:33.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.951+0000 7fd32e95d640 1 -- 172.21.15.102:0/408616483 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd32806c000 con 0x7fd328068990 2024-09-13T17:43:33.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.951+0000 7fd30ffff640 1 -- 172.21.15.102:0/408616483 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fd318013820 con 0x7fd328068990 2024-09-13T17:43:33.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.952+0000 7fd30ffff640 1 -- 172.21.15.102:0/408616483 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7fd318042630 con 0x7fd328068990 2024-09-13T17:43:33.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.952+0000 7fd32e95d640 1 -- 172.21.15.102:0/408616483 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd328064600 con 0x7fd328068990 2024-09-13T17:43:33.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.956+0000 7fd30ffff640 1 -- 172.21.15.102:0/408616483 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fd318021080 con 0x7fd328068990 2024-09-13T17:43:33.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.957+0000 7fc111ffb640 1 -- 172.21.15.102:0/3502934512 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc108001470 con 0x7fc11409c630 2024-09-13T17:43:33.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.957+0000 7fc111ffb640 1 -- 172.21.15.102:0/3502934512 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10c02d050 con 0x7fc11409c630 2024-09-13T17:43:33.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.957+0000 7fda6ffff640 1 -- 172.21.15.102:0/1653286906 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fda8000ce70 con 0x7fda8809da70 2024-09-13T17:43:33.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.957+0000 7fc111ffb640 1 -- 172.21.15.102:0/3502934512 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c031630 con 0x7fc11409c630 2024-09-13T17:43:33.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.958+0000 7fc113fff640 1 -- 172.21.15.102:0/3502934512 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc11409c630 msgr2=0x7fc114068f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.958+0000 7fc113fff640 1 --2- 172.21.15.102:0/3502934512 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc11409c630 0x7fc114068f10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc10c009e90 tx=0x7fc10c009780 comp rx=0 tx=0).stop 2024-09-13T17:43:33.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.958+0000 7fc113fff640 1 -- 172.21.15.102:0/3502934512 shutdown_connections 2024-09-13T17:43:33.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.958+0000 7fc113fff640 1 --2- 172.21.15.102:0/3502934512 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc11409c630 0x7fc114068f10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.958+0000 7fc113fff640 1 --2- 172.21.15.102:0/3502934512 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc11409bd20 0x7fc11409c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.958+0000 7fc113fff640 1 -- 172.21.15.102:0/3502934512 >> 172.21.15.102:0/3502934512 conn(0x7fc11406e860 msgr2=0x7fc11406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:33.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.958+0000 7fc113fff640 1 -- 172.21.15.102:0/3502934512 shutdown_connections 2024-09-13T17:43:33.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.958+0000 7fc113fff640 1 -- 172.21.15.102:0/3502934512 wait complete. 2024-09-13T17:43:33.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.959+0000 7fc113fff640 1 Processor -- start 2024-09-13T17:43:33.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.959+0000 7fc113fff640 1 -- start start 2024-09-13T17:43:33.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.960+0000 7fc113fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc114097650 0x7fc11411c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.960+0000 7fc113fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc11409bd20 0x7fc11411c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.960+0000 7fc113fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc11409c630 0x7fc11413e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.960+0000 7fc113fff640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11409e410 con 0x7fc11409bd20 2024-09-13T17:43:33.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.960+0000 7fc113fff640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11411ecd0 con 0x7fc114097650 2024-09-13T17:43:33.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.960+0000 7fc113fff640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc1140a1d30 con 0x7fc11409c630 2024-09-13T17:43:33.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.960+0000 7fc1127fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc11409bd20 0x7fc11411c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.960+0000 7fc112ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc114097650 0x7fc11411c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.960+0000 7fc1127fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc11409bd20 0x7fc11411c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57504/0 (socket says 172.21.15.102:57504) 2024-09-13T17:43:33.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.960+0000 7fc1127fc640 1 -- 172.21.15.102:0/1075025209 learned_addr learned my addr 172.21.15.102:0/1075025209 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:33.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.960+0000 7fc1137fe640 1 --2- 172.21.15.102:0/1075025209 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc11409c630 0x7fc11413e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.960+0000 7fc1127fc640 1 -- 172.21.15.102:0/1075025209 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc11409c630 msgr2=0x7fc11413e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.960+0000 7fc1127fc640 1 --2- 172.21.15.102:0/1075025209 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc11409c630 0x7fc11413e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.960+0000 7fc1127fc640 1 -- 172.21.15.102:0/1075025209 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc114097650 msgr2=0x7fc11411c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.960+0000 7fc1127fc640 1 --2- 172.21.15.102:0/1075025209 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc114097650 0x7fc11411c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.960+0000 7fc1127fc640 1 -- 172.21.15.102:0/1075025209 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10c009a20 con 0x7fc11409bd20 2024-09-13T17:43:33.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.960+0000 7fc1127fc640 1 --2- 172.21.15.102:0/1075025209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc11409bd20 0x7fc11411c750 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc10800cb60 tx=0x7fc108001cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:33.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.961+0000 7fc0faffd640 1 -- 172.21.15.102:0/1075025209 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc108007430 con 0x7fc11409bd20 2024-09-13T17:43:33.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.961+0000 7fc0faffd640 1 -- 172.21.15.102:0/1075025209 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc108007590 con 0x7fc11409bd20 2024-09-13T17:43:33.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.961+0000 7fc0faffd640 1 -- 172.21.15.102:0/1075025209 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1080049d0 con 0x7fc11409bd20 2024-09-13T17:43:33.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.961+0000 7fc113fff640 1 -- 172.21.15.102:0/1075025209 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11411db50 con 0x7fc11409bd20 2024-09-13T17:43:33.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.961+0000 7fc113fff640 1 -- 172.21.15.102:0/1075025209 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11411be70 con 0x7fc11409bd20 2024-09-13T17:43:33.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.962+0000 7fc0faffd640 1 -- 172.21.15.102:0/1075025209 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fc108004d40 con 0x7fc11409bd20 2024-09-13T17:43:33.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.962+0000 7fc0faffd640 1 -- 172.21.15.102:0/1075025209 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7fc108041750 con 0x7fc11409bd20 2024-09-13T17:43:33.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.962+0000 7fc113fff640 1 -- 172.21.15.102:0/1075025209 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc114064600 con 0x7fc11409bd20 2024-09-13T17:43:33.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:33.971+0000 7fc0faffd640 1 -- 172.21.15.102:0/1075025209 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fc10801b360 con 0x7fc11409bd20 2024-09-13T17:43:34.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.086+0000 7febc0b5d640 1 -- 172.21.15.102:0/3300737322 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7febbc09c7d0 con 0x7febbc068920 2024-09-13T17:43:34.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.086+0000 7febaa7fc640 1 -- 172.21.15.102:0/3300737322 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7febac021030 con 0x7febbc068920 2024-09-13T17:43:34.088 INFO:teuthology.orchestra.run.smithi102.stdout:38654705685 2024-09-13T17:43:34.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.089+0000 7febc0b5d640 1 -- 172.21.15.102:0/3300737322 >> v1:172.21.15.102:6816/34937 conn(0x7feb9c031160 legacy=0x7feb9c033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:34.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.089+0000 7febc0b5d640 1 -- 172.21.15.102:0/3300737322 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7febbc068920 msgr2=0x7febbc13ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:34.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.089+0000 7febc0b5d640 1 --2- 172.21.15.102:0/3300737322 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7febbc068920 0x7febbc13ec40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7febac00e900 tx=0x7febac0091d0 comp rx=0 tx=0).stop 2024-09-13T17:43:34.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.089+0000 7febc0b5d640 1 -- 172.21.15.102:0/3300737322 shutdown_connections 2024-09-13T17:43:34.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.089+0000 7febc0b5d640 1 --2- 172.21.15.102:0/3300737322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febbc12f310 0x7febbc11a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:34.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.089+0000 7febc0b5d640 1 --2- 172.21.15.102:0/3300737322 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7febbc09dc50 0x7febbc13f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:34.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.089+0000 7febc0b5d640 1 --2- 172.21.15.102:0/3300737322 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7febbc068920 0x7febbc13ec40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:34.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.089+0000 7febc0b5d640 1 -- 172.21.15.102:0/3300737322 >> 172.21.15.102:0/3300737322 conn(0x7febbc06f320 msgr2=0x7febbc132890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:34.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.089+0000 7febc0b5d640 1 -- 172.21.15.102:0/3300737322 shutdown_connections 2024-09-13T17:43:34.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.090+0000 7febc0b5d640 1 -- 172.21.15.102:0/3300737322 wait complete. 2024-09-13T17:43:34.105 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705686 got 38654705685 for osd.3 2024-09-13T17:43:34.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.120+0000 7fda8e2c9640 1 -- 172.21.15.102:0/1653286906 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fda88064000 con 0x7fda8809da70 2024-09-13T17:43:34.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.120+0000 7fda6ffff640 1 -- 172.21.15.102:0/1653286906 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fda8001c3c0 con 0x7fda8809da70 2024-09-13T17:43:34.122 INFO:teuthology.orchestra.run.smithi102.stdout:38654705688 2024-09-13T17:43:34.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.123+0000 7fda8e2c9640 1 -- 172.21.15.102:0/1653286906 >> v1:172.21.15.102:6816/34937 conn(0x7fda74031160 legacy=0x7fda74033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:34.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.123+0000 7fda8e2c9640 1 -- 172.21.15.102:0/1653286906 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fda8809da70 msgr2=0x7fda88122dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:34.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.123+0000 7fda8e2c9640 1 --2- 172.21.15.102:0/1653286906 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fda8809da70 0x7fda88122dc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fda80009f40 tx=0x7fda8000b040 comp rx=0 tx=0).stop 2024-09-13T17:43:34.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.123+0000 7fda8e2c9640 1 -- 172.21.15.102:0/1653286906 shutdown_connections 2024-09-13T17:43:34.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.123+0000 7fda8e2c9640 1 --2- 172.21.15.102:0/1653286906 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fda8809da70 0x7fda88122dc0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:34.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.123+0000 7fda8e2c9640 1 --2- 172.21.15.102:0/1653286906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda8806e030 0x7fda881433d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:34.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.123+0000 7fda8e2c9640 1 --2- 172.21.15.102:0/1653286906 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fda8806d720 0x7fda88142e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:34.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.123+0000 7fda8e2c9640 1 -- 172.21.15.102:0/1653286906 >> 172.21.15.102:0/1653286906 conn(0x7fda8806f320 msgr2=0x7fda8806c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:34.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.123+0000 7fda8e2c9640 1 -- 172.21.15.102:0/1653286906 shutdown_connections 2024-09-13T17:43:34.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.123+0000 7fda8e2c9640 1 -- 172.21.15.102:0/1653286906 wait complete. 2024-09-13T17:43:34.135 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705689 got 38654705688 for osd.1 2024-09-13T17:43:34.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.151+0000 7fd32e95d640 1 -- 172.21.15.102:0/408616483 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd32809c840 con 0x7fd328068990 2024-09-13T17:43:34.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.151+0000 7fd30ffff640 1 -- 172.21.15.102:0/408616483 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd318023ad0 con 0x7fd328068990 2024-09-13T17:43:34.154 INFO:teuthology.orchestra.run.smithi102.stdout:38654705686 2024-09-13T17:43:34.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.152+0000 7fc113fff640 1 -- 172.21.15.102:0/1075025209 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc11409d880 con 0x7fc11409bd20 2024-09-13T17:43:34.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.152+0000 7fc0faffd640 1 -- 172.21.15.102:0/1075025209 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc108010300 con 0x7fc11409bd20 2024-09-13T17:43:34.154 INFO:teuthology.orchestra.run.smithi102.stdout:38654705685 2024-09-13T17:43:34.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.153+0000 7fd32e95d640 1 -- 172.21.15.102:0/408616483 >> v1:172.21.15.102:6816/34937 conn(0x7fd314031160 legacy=0x7fd314033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:34.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.153+0000 7fd32e95d640 1 -- 172.21.15.102:0/408616483 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd328068990 msgr2=0x7fd32806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:34.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.153+0000 7fd32e95d640 1 --2- 172.21.15.102:0/408616483 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd328068990 0x7fd32806e320 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd31800ec80 tx=0x7fd3180091d0 comp rx=0 tx=0).stop 2024-09-13T17:43:34.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.154+0000 7fd32e95d640 1 -- 172.21.15.102:0/408616483 shutdown_connections 2024-09-13T17:43:34.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.154+0000 7fd32e95d640 1 --2- 172.21.15.102:0/408616483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd32812f410 0x7fd32806b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:34.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.154+0000 7fd32e95d640 1 --2- 172.21.15.102:0/408616483 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd32809db40 0x7fd32806aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:34.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.154+0000 7fd32e95d640 1 --2- 172.21.15.102:0/408616483 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd328068990 0x7fd32806e320 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:34.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.154+0000 7fd32e95d640 1 -- 172.21.15.102:0/408616483 >> 172.21.15.102:0/408616483 conn(0x7fd32806f480 msgr2=0x7fd3280a1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:34.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.154+0000 7fd32e95d640 1 -- 172.21.15.102:0/408616483 shutdown_connections 2024-09-13T17:43:34.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.154+0000 7fd32e95d640 1 -- 172.21.15.102:0/408616483 wait complete. 2024-09-13T17:43:34.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.154+0000 7fc113fff640 1 -- 172.21.15.102:0/1075025209 >> v1:172.21.15.102:6816/34937 conn(0x7fc100031160 legacy=0x7fc100033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:34.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.154+0000 7fc113fff640 1 -- 172.21.15.102:0/1075025209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc11409bd20 msgr2=0x7fc11411c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:34.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.154+0000 7fc113fff640 1 --2- 172.21.15.102:0/1075025209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc11409bd20 0x7fc11411c750 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc10800cb60 tx=0x7fc108001cf0 comp rx=0 tx=0).stop 2024-09-13T17:43:34.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.154+0000 7fc113fff640 1 -- 172.21.15.102:0/1075025209 shutdown_connections 2024-09-13T17:43:34.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.154+0000 7fc113fff640 1 --2- 172.21.15.102:0/1075025209 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc11409c630 0x7fc11413e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:34.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.154+0000 7fc113fff640 1 --2- 172.21.15.102:0/1075025209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc11409bd20 0x7fc11411c750 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:34.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.154+0000 7fc113fff640 1 --2- 172.21.15.102:0/1075025209 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc114097650 0x7fc11411c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:34.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.154+0000 7fc113fff640 1 -- 172.21.15.102:0/1075025209 >> 172.21.15.102:0/1075025209 conn(0x7fc11406e860 msgr2=0x7fc114132340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:34.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.154+0000 7fc113fff640 1 -- 172.21.15.102:0/1075025209 shutdown_connections 2024-09-13T17:43:34.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:34.154+0000 7fc113fff640 1 -- 172.21.15.102:0/1075025209 wait complete. 2024-09-13T17:43:34.167 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705687 got 38654705686 for osd.0 2024-09-13T17:43:34.167 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705686 got 38654705685 for osd.2 2024-09-13T17:43:35.106 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T17:43:35.137 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T17:43:35.167 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T17:43:35.168 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T17:43:35.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.225+0000 7fd83e15d640 1 Processor -- start 2024-09-13T17:43:35.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.225+0000 7fd83e15d640 1 -- start start 2024-09-13T17:43:35.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.226+0000 7fd83e15d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd83809ffb0 0x7fd83812c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.226+0000 7fd83e15d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd83812c6f0 0x7fd83812cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.226+0000 7fd83e15d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd83809a220 0x7fd83809a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.226+0000 7fd836ffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd83812c6f0 0x7fd83812cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.226+0000 7fd8377fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd83809ffb0 0x7fd83812c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.226+0000 7fd836ffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd83812c6f0 0x7fd83812cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47282/0 (socket says 172.21.15.102:47282) 2024-09-13T17:43:35.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.226+0000 7fd8377fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd83809ffb0 0x7fd83812c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:39776/0 (socket says 172.21.15.102:39776) 2024-09-13T17:43:35.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.226+0000 7fd837fff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd83809a220 0x7fd83809a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.226+0000 7fd837fff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd83809a220 0x7fd83809a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57510/0 (socket says 172.21.15.102:57510) 2024-09-13T17:43:35.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.226+0000 7fd83e15d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8380987b0 con 0x7fd83809a220 2024-09-13T17:43:35.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.226+0000 7fd836ffd640 1 -- 172.21.15.102:0/1935745275 learned_addr learned my addr 172.21.15.102:0/1935745275 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:35.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.226+0000 7fd83e15d640 1 -- 172.21.15.102:0/1935745275 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83806ece0 con 0x7fd83809ffb0 2024-09-13T17:43:35.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.226+0000 7fd83e15d640 1 -- 172.21.15.102:0/1935745275 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd83812d000 con 0x7fd83812c6f0 2024-09-13T17:43:35.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.227+0000 7fd836ffd640 1 -- 172.21.15.102:0/1935745275 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd83809ffb0 msgr2=0x7fd83812c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:35.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.227+0000 7fd836ffd640 1 --2- 172.21.15.102:0/1935745275 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd83809ffb0 0x7fd83812c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.227+0000 7fd836ffd640 1 -- 172.21.15.102:0/1935745275 >> v2:172.21.15.102:3300/0 conn(0x7fd83809a220 msgr2=0x7fd83809a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:35.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.227+0000 7fd836ffd640 1 --2- 172.21.15.102:0/1935745275 >> v2:172.21.15.102:3300/0 conn(0x7fd83809a220 0x7fd83809a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.227+0000 7fd836ffd640 1 -- 172.21.15.102:0/1935745275 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd838097890 con 0x7fd83812c6f0 2024-09-13T17:43:35.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.227+0000 7fd836ffd640 1 --2- 172.21.15.102:0/1935745275 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd83812c6f0 0x7fd83812cac0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009940 tx=0x7fd82c0091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae8db51f78a839d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:35.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.228+0000 7fd8367fc640 1 -- 172.21.15.102:0/1935745275 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c02ad00 con 0x7fd83812c6f0 2024-09-13T17:43:35.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.228+0000 7fd8367fc640 1 -- 172.21.15.102:0/1935745275 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82c02ae60 con 0x7fd83812c6f0 2024-09-13T17:43:35.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.228+0000 7fd8367fc640 1 -- 172.21.15.102:0/1935745275 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c032670 con 0x7fd83812c6f0 2024-09-13T17:43:35.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.229+0000 7fd83e15d640 1 -- 172.21.15.102:0/1935745275 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd83812c6f0 msgr2=0x7fd83812cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.229+0000 7fd83e15d640 1 --2- 172.21.15.102:0/1935745275 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd83812c6f0 0x7fd83812cac0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009940 tx=0x7fd82c0091d0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.229+0000 7fd83e15d640 1 -- 172.21.15.102:0/1935745275 shutdown_connections 2024-09-13T17:43:35.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.229+0000 7fd83e15d640 1 --2- 172.21.15.102:0/1935745275 >> v2:172.21.15.102:3300/0 conn(0x7fd83809a220 0x7fd83809a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.229+0000 7fd83e15d640 1 --2- 172.21.15.102:0/1935745275 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd83812c6f0 0x7fd83812cac0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.229+0000 7fd83e15d640 1 --2- 172.21.15.102:0/1935745275 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd83809ffb0 0x7fd83812c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.229+0000 7fd83e15d640 1 -- 172.21.15.102:0/1935745275 >> 172.21.15.102:0/1935745275 conn(0x7fd83805e130 msgr2=0x7fd83809f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:35.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.229+0000 7fd83e15d640 1 -- 172.21.15.102:0/1935745275 shutdown_connections 2024-09-13T17:43:35.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.229+0000 7fd83e15d640 1 -- 172.21.15.102:0/1935745275 wait complete. 2024-09-13T17:43:35.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.230+0000 7fd83e15d640 1 Processor -- start 2024-09-13T17:43:35.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.231+0000 7fd83e15d640 1 -- start start 2024-09-13T17:43:35.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.231+0000 7fd83e15d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd83809a220 0x7fd83811a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.231+0000 7fd83e15d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd83809ffb0 0x7fd83811af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.231+0000 7fd83e15d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd83812c6f0 0x7fd83811b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.231+0000 7fd8377fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd83809a220 0x7fd83811a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.231+0000 7fd8377fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd83809a220 0x7fd83811a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:39792/0 (socket says 172.21.15.102:39792) 2024-09-13T17:43:35.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.231+0000 7fd837fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd83812c6f0 0x7fd83811b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.231+0000 7fd836ffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd83809ffb0 0x7fd83811af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.232+0000 7fd837fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd83812c6f0 0x7fd83811b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57518/0 (socket says 172.21.15.102:57518) 2024-09-13T17:43:35.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.232+0000 7fd836ffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd83809ffb0 0x7fd83811af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47290/0 (socket says 172.21.15.102:47290) 2024-09-13T17:43:35.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.232+0000 7fd8377fe640 1 -- 172.21.15.102:0/1391215030 learned_addr learned my addr 172.21.15.102:0/1391215030 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:35.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.232+0000 7fd83e15d640 1 -- 172.21.15.102:0/1391215030 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83812b840 con 0x7fd83812c6f0 2024-09-13T17:43:35.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.232+0000 7fd83e15d640 1 -- 172.21.15.102:0/1391215030 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd838096e30 con 0x7fd83809a220 2024-09-13T17:43:35.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.232+0000 7fd83e15d640 1 -- 172.21.15.102:0/1391215030 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd83809e7e0 con 0x7fd83809ffb0 2024-09-13T17:43:35.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.232+0000 7fd836ffd640 1 -- 172.21.15.102:0/1391215030 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd83809a220 msgr2=0x7fd83811a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.232+0000 7fd836ffd640 1 --2- 172.21.15.102:0/1391215030 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd83809a220 0x7fd83811a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.232+0000 7fd836ffd640 1 -- 172.21.15.102:0/1391215030 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd83812c6f0 msgr2=0x7fd83811b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.232+0000 7fd836ffd640 1 --2- 172.21.15.102:0/1391215030 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd83812c6f0 0x7fd83811b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.232+0000 7fd836ffd640 1 -- 172.21.15.102:0/1391215030 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd82c009450 con 0x7fd83809ffb0 2024-09-13T17:43:35.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.232+0000 7fd837fff640 1 --2- 172.21.15.102:0/1391215030 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd83812c6f0 0x7fd83811b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:35.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.232+0000 7fd8377fe640 1 --2- 172.21.15.102:0/1391215030 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd83809a220 0x7fd83811a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:35.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.232+0000 7fd836ffd640 1 --2- 172.21.15.102:0/1391215030 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd83809ffb0 0x7fd83811af10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd82c031000 tx=0x7fd82c032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:35.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.233+0000 7fd81f7fe640 1 -- 172.21.15.102:0/1391215030 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c03b610 con 0x7fd83809ffb0 2024-09-13T17:43:35.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.233+0000 7fd81f7fe640 1 -- 172.21.15.102:0/1391215030 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82c03b790 con 0x7fd83809ffb0 2024-09-13T17:43:35.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.233+0000 7fd81f7fe640 1 -- 172.21.15.102:0/1391215030 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c03bb50 con 0x7fd83809ffb0 2024-09-13T17:43:35.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.233+0000 7fd83e15d640 1 -- 172.21.15.102:0/1391215030 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd838121cd0 con 0x7fd83809ffb0 2024-09-13T17:43:35.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.233+0000 7fd83e15d640 1 -- 172.21.15.102:0/1391215030 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd838067bb0 con 0x7fd83809ffb0 2024-09-13T17:43:35.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.234+0000 7fd81f7fe640 1 -- 172.21.15.102:0/1391215030 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fd82c039040 con 0x7fd83809ffb0 2024-09-13T17:43:35.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.234+0000 7fd83e15d640 1 -- 172.21.15.102:0/1391215030 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd838064600 con 0x7fd83809ffb0 2024-09-13T17:43:35.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.234+0000 7fd81f7fe640 1 -- 172.21.15.102:0/1391215030 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7fd82c062470 con 0x7fd83809ffb0 2024-09-13T17:43:35.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.239+0000 7fd81f7fe640 1 -- 172.21.15.102:0/1391215030 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fd82c044af0 con 0x7fd83809ffb0 2024-09-13T17:43:35.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.264+0000 7f65dd434640 1 Processor -- start 2024-09-13T17:43:35.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.264+0000 7f65dd434640 1 -- start start 2024-09-13T17:43:35.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.264+0000 7f65dd434640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65d809f6e0 0x7f65d806e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.264+0000 7f65dd434640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65d806e0b0 0x7f65d812d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.264+0000 7f65dd434640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65d809fe10 con 0x7f65d812d7c0 2024-09-13T17:43:35.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.264+0000 7f65dd434640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d809fab0 con 0x7f65d809f6e0 2024-09-13T17:43:35.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.264+0000 7f65dd434640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f65d806c070 con 0x7f65d806e0b0 2024-09-13T17:43:35.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.264+0000 7f65d67fc640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65d806e0b0 0x7f65d812d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.265+0000 7f65d67fc640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65d806e0b0 0x7f65d812d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47292/0 (socket says 172.21.15.102:47292) 2024-09-13T17:43:35.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.264+0000 7f65d77fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f65d812d7c0 0x7f65d81345c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:56250/0 (socket says 172.21.15.102:56250) 2024-09-13T17:43:35.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.265+0000 7f65d67fc640 1 -- 172.21.15.102:0/2535016230 learned_addr learned my addr 172.21.15.102:0/2535016230 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:35.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.265+0000 7f65d6ffd640 1 --2- 172.21.15.102:0/2535016230 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65d809f6e0 0x7f65d806e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.265+0000 7f65d67fc640 1 -- 172.21.15.102:0/2535016230 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65d809f6e0 msgr2=0x7f65d806e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:35.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.265+0000 7f65d67fc640 1 --2- 172.21.15.102:0/2535016230 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65d809f6e0 0x7f65d806e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.265+0000 7f65d67fc640 1 -- 172.21.15.102:0/2535016230 >> v1:172.21.15.102:6789/0 conn(0x7f65d812d7c0 legacy=0x7f65d81345c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.265+0000 7f65d67fc640 1 -- 172.21.15.102:0/2535016230 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d80a0440 con 0x7f65d806e0b0 2024-09-13T17:43:35.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.266+0000 7f65d67fc640 1 --2- 172.21.15.102:0/2535016230 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65d806e0b0 0x7f65d812d280 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f65cc009940 tx=0x7f65cc0091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a0d99e65db0c2b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:35.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.266+0000 7f65d5ffb640 1 -- 172.21.15.102:0/2535016230 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc02ad00 con 0x7f65d806e0b0 2024-09-13T17:43:35.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.266+0000 7f65d5ffb640 1 -- 172.21.15.102:0/2535016230 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65cc02ae60 con 0x7f65d806e0b0 2024-09-13T17:43:35.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.266+0000 7f65d5ffb640 1 -- 172.21.15.102:0/2535016230 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc032670 con 0x7f65d806e0b0 2024-09-13T17:43:35.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.266+0000 7f65dd434640 1 -- 172.21.15.102:0/2535016230 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65d806e0b0 msgr2=0x7f65d812d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.266+0000 7f65dd434640 1 --2- 172.21.15.102:0/2535016230 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65d806e0b0 0x7f65d812d280 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f65cc009940 tx=0x7f65cc0091d0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.267+0000 7f65dd434640 1 -- 172.21.15.102:0/2535016230 shutdown_connections 2024-09-13T17:43:35.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.267+0000 7f65dd434640 1 --2- 172.21.15.102:0/2535016230 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65d806e0b0 0x7f65d812d280 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.267+0000 7f65dd434640 1 --2- 172.21.15.102:0/2535016230 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65d809f6e0 0x7f65d806e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.267+0000 7f65dd434640 1 -- 172.21.15.102:0/2535016230 >> 172.21.15.102:0/2535016230 conn(0x7f65d8067bf0 msgr2=0x7f65d80a07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:35.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.267+0000 7f65dd434640 1 -- 172.21.15.102:0/2535016230 shutdown_connections 2024-09-13T17:43:35.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.267+0000 7f65dd434640 1 -- 172.21.15.102:0/2535016230 wait complete. 2024-09-13T17:43:35.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.268+0000 7f65dd434640 1 Processor -- start 2024-09-13T17:43:35.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.268+0000 7f65dd434640 1 -- start start 2024-09-13T17:43:35.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.268+0000 7f65dd434640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65d806e0b0 0x7f65d811bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.268+0000 7f65dd434640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65d809f6e0 0x7f65d811c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.268+0000 7f65dd434640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65d812d7c0 0x7f65d811c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.268+0000 7f65dd434640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d811cf80 con 0x7f65d809f6e0 2024-09-13T17:43:35.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.268+0000 7f65dd434640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d809d160 con 0x7f65d806e0b0 2024-09-13T17:43:35.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.268+0000 7f65dd434640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f65d806c1c0 con 0x7f65d812d7c0 2024-09-13T17:43:35.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.268+0000 7f65d6ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65d806e0b0 0x7f65d811bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.268+0000 7f65d77fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65d812d7c0 0x7f65d811c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.268+0000 7f65d6ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65d806e0b0 0x7f65d811bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:39816/0 (socket says 172.21.15.102:39816) 2024-09-13T17:43:35.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.268+0000 7f65d6ffd640 1 -- 172.21.15.102:0/1514312112 learned_addr learned my addr 172.21.15.102:0/1514312112 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:35.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.269+0000 7f65d67fc640 1 --2- 172.21.15.102:0/1514312112 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65d809f6e0 0x7f65d811c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.269+0000 7f65d77fe640 1 -- 172.21.15.102:0/1514312112 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65d806e0b0 msgr2=0x7f65d811bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.269+0000 7f65d77fe640 1 --2- 172.21.15.102:0/1514312112 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65d806e0b0 0x7f65d811bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.269+0000 7f65d77fe640 1 -- 172.21.15.102:0/1514312112 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65d809f6e0 msgr2=0x7f65d811c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.269+0000 7f65d77fe640 1 --2- 172.21.15.102:0/1514312112 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65d809f6e0 0x7f65d811c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.269+0000 7f65d77fe640 1 -- 172.21.15.102:0/1514312112 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65cc009450 con 0x7f65d812d7c0 2024-09-13T17:43:35.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.269+0000 7f65d6ffd640 1 --2- 172.21.15.102:0/1514312112 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65d806e0b0 0x7f65d811bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:35.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.269+0000 7f65d77fe640 1 --2- 172.21.15.102:0/1514312112 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65d812d7c0 0x7f65d811c590 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f65d0001f00 tx=0x7f65d000e630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:35.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.269+0000 7f65beffd640 1 -- 172.21.15.102:0/1514312112 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d0013070 con 0x7f65d812d7c0 2024-09-13T17:43:35.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.269+0000 7f65beffd640 1 -- 172.21.15.102:0/1514312112 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d000e0d0 con 0x7f65d812d7c0 2024-09-13T17:43:35.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.269+0000 7f65beffd640 1 -- 172.21.15.102:0/1514312112 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d0004d90 con 0x7f65d812d7c0 2024-09-13T17:43:35.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.269+0000 7f65dd434640 1 -- 172.21.15.102:0/1514312112 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d811faf0 con 0x7f65d812d7c0 2024-09-13T17:43:35.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.269+0000 7f65dd434640 1 -- 172.21.15.102:0/1514312112 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d811f250 con 0x7f65d812d7c0 2024-09-13T17:43:35.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.271+0000 7f65beffd640 1 -- 172.21.15.102:0/1514312112 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f65d0010cd0 con 0x7f65d812d7c0 2024-09-13T17:43:35.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.271+0000 7f65dd434640 1 -- 172.21.15.102:0/1514312112 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d8064000 con 0x7f65d812d7c0 2024-09-13T17:43:35.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.271+0000 7f65beffd640 1 -- 172.21.15.102:0/1514312112 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f65d0045d30 con 0x7f65d812d7c0 2024-09-13T17:43:35.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.279+0000 7f65beffd640 1 -- 172.21.15.102:0/1514312112 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f65d00070f0 con 0x7f65d812d7c0 2024-09-13T17:43:35.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.299+0000 7f2efdd1d640 1 Processor -- start 2024-09-13T17:43:35.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.299+0000 7f2efdd1d640 1 -- start start 2024-09-13T17:43:35.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.300+0000 7f2efdd1d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2ef8068920 0x7f2ef809d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.300+0000 7f2efdd1d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ef809dc50 0x7f2ef809e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.300+0000 7f2efdd1d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ef812f310 0x7f2ef812f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.300+0000 7f2efdd1d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ef8068cf0 con 0x7f2ef8068920 2024-09-13T17:43:35.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.300+0000 7f2efdd1d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef80695a0 con 0x7f2ef809dc50 2024-09-13T17:43:35.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.300+0000 7f2efdd1d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef809f090 con 0x7f2ef812f310 2024-09-13T17:43:35.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.300+0000 7f2ef7fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ef812f310 0x7f2ef812f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.300+0000 7f2ef6ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ef809dc50 0x7f2ef809e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.300+0000 7f2ef7fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ef812f310 0x7f2ef812f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47312/0 (socket says 172.21.15.102:47312) 2024-09-13T17:43:35.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.300+0000 7f2ef7fff640 1 -- 172.21.15.102:0/4060197164 learned_addr learned my addr 172.21.15.102:0/4060197164 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:35.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.300+0000 7f2ef77fe640 1 --2- 172.21.15.102:0/4060197164 >> v2:172.21.15.102:3300/0 conn(0x7f2ef8068920 0x7f2ef809d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.301+0000 7f2ef6ffd640 1 -- 172.21.15.102:0/4060197164 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ef812f310 msgr2=0x7f2ef812f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:35.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.301+0000 7f2ef6ffd640 1 --2- 172.21.15.102:0/4060197164 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ef812f310 0x7f2ef812f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.301+0000 7f2ef6ffd640 1 -- 172.21.15.102:0/4060197164 >> v2:172.21.15.102:3300/0 conn(0x7f2ef8068920 msgr2=0x7f2ef809d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:35.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.301+0000 7f2ef6ffd640 1 --2- 172.21.15.102:0/4060197164 >> v2:172.21.15.102:3300/0 conn(0x7f2ef8068920 0x7f2ef809d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.301+0000 7f2ef6ffd640 1 -- 172.21.15.102:0/4060197164 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef809cce0 con 0x7f2ef809dc50 2024-09-13T17:43:35.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.301+0000 7f2ef7fff640 1 --2- 172.21.15.102:0/4060197164 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ef812f310 0x7f2ef812f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:35.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.301+0000 7f2ef77fe640 1 --2- 172.21.15.102:0/4060197164 >> v2:172.21.15.102:3300/0 conn(0x7f2ef8068920 0x7f2ef809d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:35.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.301+0000 7f2ef6ffd640 1 --2- 172.21.15.102:0/4060197164 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ef809dc50 0x7f2ef809e020 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2eec009ab0 tx=0x7f2eec0090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1963c93166f6e6e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:35.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.302+0000 7f2ef67fc640 1 -- 172.21.15.102:0/4060197164 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec02aca0 con 0x7f2ef809dc50 2024-09-13T17:43:35.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.302+0000 7f2ef67fc640 1 -- 172.21.15.102:0/4060197164 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eec02ae00 con 0x7f2ef809dc50 2024-09-13T17:43:35.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.302+0000 7f2ef67fc640 1 -- 172.21.15.102:0/4060197164 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec032670 con 0x7f2ef809dc50 2024-09-13T17:43:35.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.302+0000 7f7dd4b1d640 1 Processor -- start 2024-09-13T17:43:35.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.302+0000 7f2efdd1d640 1 -- 172.21.15.102:0/4060197164 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ef809dc50 msgr2=0x7f2ef809e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.302+0000 7f7dd4b1d640 1 -- start start 2024-09-13T17:43:35.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.302+0000 7f2efdd1d640 1 --2- 172.21.15.102:0/4060197164 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ef809dc50 0x7f2ef809e020 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2eec009ab0 tx=0x7f2eec0090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.303+0000 7f2efdd1d640 1 -- 172.21.15.102:0/4060197164 shutdown_connections 2024-09-13T17:43:35.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.303+0000 7f2efdd1d640 1 --2- 172.21.15.102:0/4060197164 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ef812f310 0x7f2ef812f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.303+0000 7f2efdd1d640 1 --2- 172.21.15.102:0/4060197164 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ef809dc50 0x7f2ef809e020 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.303+0000 7f2efdd1d640 1 --2- 172.21.15.102:0/4060197164 >> v2:172.21.15.102:3300/0 conn(0x7f2ef8068920 0x7f2ef809d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.303+0000 7f2efdd1d640 1 -- 172.21.15.102:0/4060197164 >> 172.21.15.102:0/4060197164 conn(0x7f2ef806f320 msgr2=0x7f2ef80a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:35.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.303+0000 7f2efdd1d640 1 -- 172.21.15.102:0/4060197164 shutdown_connections 2024-09-13T17:43:35.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.303+0000 7f2efdd1d640 1 -- 172.21.15.102:0/4060197164 wait complete. 2024-09-13T17:43:35.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.303+0000 7f7dd4b1d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7dd006d670 0x7f7dd006e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.303+0000 7f7dd4b1d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f7dd009f7f0 0x7f7dd012da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.303+0000 7f7dce59c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7dd006d670 0x7f7dd006e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.303+0000 7f7dd4b1d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dd006c910 con 0x7f7dd006eb60 2024-09-13T17:43:35.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.303+0000 7f7dd4b1d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dd006da40 con 0x7f7dd009f7f0 2024-09-13T17:43:35.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.303+0000 7f7dd4b1d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd006c130 con 0x7f7dd006d670 2024-09-13T17:43:35.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.303+0000 7f7dce59c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7dd006d670 0x7f7dd006e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:39844/0 (socket says 172.21.15.102:39844) 2024-09-13T17:43:35.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.303+0000 7f7dced9d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f7dd009f7f0 0x7f7dd012da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.303+0000 7f7dce59c640 1 -- 172.21.15.102:0/3680602196 learned_addr learned my addr 172.21.15.102:0/3680602196 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:35.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.304+0000 7f2efdd1d640 1 Processor -- start 2024-09-13T17:43:35.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.304+0000 7f2efdd1d640 1 -- start start 2024-09-13T17:43:35.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.304+0000 7f7dced9d640 1 -- 172.21.15.102:0/3680602196 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7dd006d670 msgr2=0x7f7dd006e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:35.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.304+0000 7f7dced9d640 1 --2- 172.21.15.102:0/3680602196 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7dd006d670 0x7f7dd006e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.304+0000 7f7dced9d640 1 -- 172.21.15.102:0/3680602196 >> v1:172.21.15.102:6789/0 conn(0x7f7dd006eb60 legacy=0x7f7dd012d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.304+0000 7f7dced9d640 1 -- 172.21.15.102:0/3680602196 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd00a0310 con 0x7f7dd009f7f0 2024-09-13T17:43:35.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.304+0000 7f2efdd1d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ef8068920 0x7f2ef811dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.304+0000 7f2efdd1d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ef809dc50 0x7f2ef811e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.304+0000 7f2efdd1d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ef812f310 0x7f2ef811e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.304+0000 7f2ef6ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ef809dc50 0x7f2ef811e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.304+0000 7f2ef77fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ef8068920 0x7f2ef811dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.304+0000 7f2ef77fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ef8068920 0x7f2ef811dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:39856/0 (socket says 172.21.15.102:39856) 2024-09-13T17:43:35.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.304+0000 7f2ef6ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ef809dc50 0x7f2ef811e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57572/0 (socket says 172.21.15.102:57572) 2024-09-13T17:43:35.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.305+0000 7f2efdd1d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef809ffd0 con 0x7f2ef809dc50 2024-09-13T17:43:35.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.305+0000 7f2efdd1d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef809e620 con 0x7f2ef8068920 2024-09-13T17:43:35.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.305+0000 7f2efdd1d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef811f1f0 con 0x7f2ef812f310 2024-09-13T17:43:35.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.305+0000 7f2ef77fe640 1 -- 172.21.15.102:0/552244492 learned_addr learned my addr 172.21.15.102:0/552244492 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:35.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.305+0000 7f2ef7fff640 1 --2- 172.21.15.102:0/552244492 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ef812f310 0x7f2ef811e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.305+0000 7f2ef77fe640 1 -- 172.21.15.102:0/552244492 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ef812f310 msgr2=0x7f2ef811e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.305+0000 7f2ef77fe640 1 --2- 172.21.15.102:0/552244492 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ef812f310 0x7f2ef811e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.305+0000 7f2ef77fe640 1 -- 172.21.15.102:0/552244492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ef809dc50 msgr2=0x7f2ef811e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.305+0000 7f2ef77fe640 1 --2- 172.21.15.102:0/552244492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ef809dc50 0x7f2ef811e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.305+0000 7f2ef77fe640 1 -- 172.21.15.102:0/552244492 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee8009390 con 0x7f2ef8068920 2024-09-13T17:43:35.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.305+0000 7f7dced9d640 1 --2- 172.21.15.102:0/3680602196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dd009f7f0 0x7f7dd012da30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8009e60 tx=0x7f7dc8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3de7a24d6d30363 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:35.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.305+0000 7f2ef77fe640 1 --2- 172.21.15.102:0/552244492 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ef8068920 0x7f2ef811dd80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2ee800eaa0 tx=0x7f2ee80090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:35.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.305+0000 7f7dcdd9b640 1 -- 172.21.15.102:0/3680602196 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8007550 con 0x7f7dd009f7f0 2024-09-13T17:43:35.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.305+0000 7f2edf7fe640 1 -- 172.21.15.102:0/552244492 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee800ca10 con 0x7f2ef8068920 2024-09-13T17:43:35.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.305+0000 7f2efdd1d640 1 -- 172.21.15.102:0/552244492 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef0009ae0 con 0x7f2ef8068920 2024-09-13T17:43:35.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.305+0000 7f7dcdd9b640 1 -- 172.21.15.102:0/3680602196 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc802d050 con 0x7f7dd009f7f0 2024-09-13T17:43:35.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.305+0000 7f7dcdd9b640 1 -- 172.21.15.102:0/3680602196 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc80078e0 con 0x7f7dd009f7f0 2024-09-13T17:43:35.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.305+0000 7f2efdd1d640 1 -- 172.21.15.102:0/552244492 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eec009390 con 0x7f2ef8068920 2024-09-13T17:43:35.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.305+0000 7f2edf7fe640 1 -- 172.21.15.102:0/552244492 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee800cb70 con 0x7f2ef8068920 2024-09-13T17:43:35.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.305+0000 7f2edf7fe640 1 -- 172.21.15.102:0/552244492 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8013470 con 0x7f2ef8068920 2024-09-13T17:43:35.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.306+0000 7f7dd4b1d640 1 -- 172.21.15.102:0/3680602196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dd009f7f0 msgr2=0x7f7dd012da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.306+0000 7f7dd4b1d640 1 --2- 172.21.15.102:0/3680602196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dd009f7f0 0x7f7dd012da30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8009e60 tx=0x7f7dc8009780 comp rx=0 tx=0).stop 2024-09-13T17:43:35.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.306+0000 7f7dd4b1d640 1 -- 172.21.15.102:0/3680602196 shutdown_connections 2024-09-13T17:43:35.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.306+0000 7f7dd4b1d640 1 --2- 172.21.15.102:0/3680602196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dd009f7f0 0x7f7dd012da30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.306+0000 7f7dd4b1d640 1 --2- 172.21.15.102:0/3680602196 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7dd006d670 0x7f7dd006e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.306+0000 7f7dd4b1d640 1 -- 172.21.15.102:0/3680602196 >> 172.21.15.102:0/3680602196 conn(0x7f7dd0067bb0 msgr2=0x7f7dd00a08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:35.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.306+0000 7f7dd4b1d640 1 -- 172.21.15.102:0/3680602196 shutdown_connections 2024-09-13T17:43:35.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.306+0000 7f7dd4b1d640 1 -- 172.21.15.102:0/3680602196 wait complete. 2024-09-13T17:43:35.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.307+0000 7f2edf7fe640 1 -- 172.21.15.102:0/552244492 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f2ee8013820 con 0x7f2ef8068920 2024-09-13T17:43:35.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.307+0000 7f2edf7fe640 1 -- 172.21.15.102:0/552244492 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f2ee8042660 con 0x7f2ef8068920 2024-09-13T17:43:35.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.307+0000 7f2efdd1d640 1 -- 172.21.15.102:0/552244492 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ef8064000 con 0x7f2ef8068920 2024-09-13T17:43:35.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.307+0000 7f7dd4b1d640 1 Processor -- start 2024-09-13T17:43:35.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.308+0000 7f7dd4b1d640 1 -- start start 2024-09-13T17:43:35.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.308+0000 7f7dd4b1d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7dd006d670 0x7f7dd011bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.308+0000 7f7dd4b1d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dd006eb60 0x7f7dd011c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.308+0000 7f7dd4b1d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7dd009f7f0 0x7f7dd011c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.308+0000 7f7dc7fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dd006eb60 0x7f7dd011c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.308+0000 7f7dc7fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dd006eb60 0x7f7dd011c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57584/0 (socket says 172.21.15.102:57584) 2024-09-13T17:43:35.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.308+0000 7f7dce59c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7dd006d670 0x7f7dd011bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.308+0000 7f7dc7fff640 1 -- 172.21.15.102:0/3964785251 learned_addr learned my addr 172.21.15.102:0/3964785251 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:35.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.308+0000 7f7dd4b1d640 1 -- 172.21.15.102:0/3964785251 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd009d820 con 0x7f7dd006eb60 2024-09-13T17:43:35.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.308+0000 7f7dd4b1d640 1 -- 172.21.15.102:0/3964785251 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd006c290 con 0x7f7dd009f7f0 2024-09-13T17:43:35.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.308+0000 7f7dd4b1d640 1 -- 172.21.15.102:0/3964785251 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd011ebd0 con 0x7f7dd006d670 2024-09-13T17:43:35.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.308+0000 7f7dced9d640 1 --2- 172.21.15.102:0/3964785251 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7dd009f7f0 0x7f7dd011c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.309+0000 7f7dc7fff640 1 -- 172.21.15.102:0/3964785251 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7dd006d670 msgr2=0x7f7dd011bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.309+0000 7f7dc7fff640 1 --2- 172.21.15.102:0/3964785251 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7dd006d670 0x7f7dd011bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.309+0000 7f7dc7fff640 1 -- 172.21.15.102:0/3964785251 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7dd009f7f0 msgr2=0x7f7dd011c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.309+0000 7f7dc7fff640 1 --2- 172.21.15.102:0/3964785251 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7dd009f7f0 0x7f7dd011c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.309+0000 7f7dc7fff640 1 -- 172.21.15.102:0/3964785251 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc8009a20 con 0x7f7dd006eb60 2024-09-13T17:43:35.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.309+0000 7f7dced9d640 1 --2- 172.21.15.102:0/3964785251 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7dd009f7f0 0x7f7dd011c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:35.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.309+0000 7f7dc7fff640 1 --2- 172.21.15.102:0/3964785251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dd006eb60 0x7f7dd011c260 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7db80018b0 tx=0x7f7db8007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:35.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.309+0000 7f7dc67fc640 1 -- 172.21.15.102:0/3964785251 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db8010330 con 0x7f7dd006eb60 2024-09-13T17:43:35.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.309+0000 7f7dc67fc640 1 -- 172.21.15.102:0/3964785251 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db800a5e0 con 0x7f7dd006eb60 2024-09-13T17:43:35.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.309+0000 7f7dc67fc640 1 -- 172.21.15.102:0/3964785251 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db8010490 con 0x7f7dd006eb60 2024-09-13T17:43:35.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.309+0000 7f7dd4b1d640 1 -- 172.21.15.102:0/3964785251 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dd011d2d0 con 0x7f7dd006eb60 2024-09-13T17:43:35.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.310+0000 7f7dd4b1d640 1 -- 172.21.15.102:0/3964785251 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dd011f9d0 con 0x7f7dd006eb60 2024-09-13T17:43:35.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.311+0000 7f7dc67fc640 1 -- 172.21.15.102:0/3964785251 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f7db800ac30 con 0x7f7dd006eb60 2024-09-13T17:43:35.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.311+0000 7f7dc67fc640 1 -- 172.21.15.102:0/3964785251 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f7db8045fb0 con 0x7f7dd006eb60 2024-09-13T17:43:35.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.311+0000 7f7dd4b1d640 1 -- 172.21.15.102:0/3964785251 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dd0064600 con 0x7f7dd006eb60 2024-09-13T17:43:35.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.312+0000 7f2edf7fe640 1 -- 172.21.15.102:0/552244492 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f2ee8021070 con 0x7f2ef8068920 2024-09-13T17:43:35.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.319+0000 7f7dc67fc640 1 -- 172.21.15.102:0/3964785251 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f7db8016d60 con 0x7f7dd006eb60 2024-09-13T17:43:35.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.412+0000 7fd83e15d640 1 -- 172.21.15.102:0/1391215030 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd838099a20 con 0x7fd83809ffb0 2024-09-13T17:43:35.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.413+0000 7fd81f7fe640 1 -- 172.21.15.102:0/1391215030 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd82c037090 con 0x7fd83809ffb0 2024-09-13T17:43:35.415 INFO:teuthology.orchestra.run.smithi102.stdout:38654705686 2024-09-13T17:43:35.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.416+0000 7fd83e15d640 1 -- 172.21.15.102:0/1391215030 >> v1:172.21.15.102:6816/34937 conn(0x7fd824031160 legacy=0x7fd824033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.416+0000 7fd83e15d640 1 -- 172.21.15.102:0/1391215030 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd83809ffb0 msgr2=0x7fd83811af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.416+0000 7fd83e15d640 1 --2- 172.21.15.102:0/1391215030 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd83809ffb0 0x7fd83811af10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd82c031000 tx=0x7fd82c032000 comp rx=0 tx=0).stop 2024-09-13T17:43:35.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.416+0000 7fd83e15d640 1 -- 172.21.15.102:0/1391215030 shutdown_connections 2024-09-13T17:43:35.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.416+0000 7fd83e15d640 1 --2- 172.21.15.102:0/1391215030 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd83812c6f0 0x7fd83811b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.416+0000 7fd83e15d640 1 --2- 172.21.15.102:0/1391215030 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd83809ffb0 0x7fd83811af10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.416+0000 7fd83e15d640 1 --2- 172.21.15.102:0/1391215030 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd83809a220 0x7fd83811a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.416+0000 7fd83e15d640 1 -- 172.21.15.102:0/1391215030 >> 172.21.15.102:0/1391215030 conn(0x7fd83805e130 msgr2=0x7fd83806d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:35.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.416+0000 7fd83e15d640 1 -- 172.21.15.102:0/1391215030 shutdown_connections 2024-09-13T17:43:35.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.416+0000 7fd83e15d640 1 -- 172.21.15.102:0/1391215030 wait complete. 2024-09-13T17:43:35.428 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705686 got 38654705686 for osd.3 2024-09-13T17:43:35.429 DEBUG:teuthology.parallel:result is None 2024-09-13T17:43:35.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.450+0000 7f65dd434640 1 -- 172.21.15.102:0/1514312112 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f65d809af10 con 0x7f65d812d7c0 2024-09-13T17:43:35.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.451+0000 7f65beffd640 1 -- 172.21.15.102:0/1514312112 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f65d0044a50 con 0x7f65d812d7c0 2024-09-13T17:43:35.453 INFO:teuthology.orchestra.run.smithi102.stdout:38654705689 2024-09-13T17:43:35.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.453+0000 7f65dd434640 1 -- 172.21.15.102:0/1514312112 >> v1:172.21.15.102:6816/34937 conn(0x7f65c4031160 legacy=0x7f65c4033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.453+0000 7f65dd434640 1 -- 172.21.15.102:0/1514312112 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65d812d7c0 msgr2=0x7f65d811c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.453+0000 7f65dd434640 1 --2- 172.21.15.102:0/1514312112 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65d812d7c0 0x7f65d811c590 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f65d0001f00 tx=0x7f65d000e630 comp rx=0 tx=0).stop 2024-09-13T17:43:35.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.453+0000 7f65dd434640 1 -- 172.21.15.102:0/1514312112 shutdown_connections 2024-09-13T17:43:35.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.453+0000 7f65dd434640 1 --2- 172.21.15.102:0/1514312112 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65d812d7c0 0x7f65d811c590 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.453+0000 7f65dd434640 1 --2- 172.21.15.102:0/1514312112 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65d809f6e0 0x7f65d811c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.453+0000 7f65dd434640 1 --2- 172.21.15.102:0/1514312112 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65d806e0b0 0x7f65d811bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.453+0000 7f65dd434640 1 -- 172.21.15.102:0/1514312112 >> 172.21.15.102:0/1514312112 conn(0x7f65d8067bf0 msgr2=0x7f65d809a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:35.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.453+0000 7f65dd434640 1 -- 172.21.15.102:0/1514312112 shutdown_connections 2024-09-13T17:43:35.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.453+0000 7f65dd434640 1 -- 172.21.15.102:0/1514312112 wait complete. 2024-09-13T17:43:35.465 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705689 got 38654705689 for osd.1 2024-09-13T17:43:35.465 DEBUG:teuthology.parallel:result is None 2024-09-13T17:43:35.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.483+0000 7f2efdd1d640 1 -- 172.21.15.102:0/552244492 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2ef809c7d0 con 0x7f2ef8068920 2024-09-13T17:43:35.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.483+0000 7f2edf7fe640 1 -- 172.21.15.102:0/552244492 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2ee8023ad0 con 0x7f2ef8068920 2024-09-13T17:43:35.485 INFO:teuthology.orchestra.run.smithi102.stdout:38654705686 2024-09-13T17:43:35.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.483+0000 7f7dd4b1d640 1 -- 172.21.15.102:0/3964785251 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7dd009af10 con 0x7f7dd006eb60 2024-09-13T17:43:35.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.483+0000 7f7dc67fc640 1 -- 172.21.15.102:0/3964785251 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7db8016940 con 0x7f7dd006eb60 2024-09-13T17:43:35.486 INFO:teuthology.orchestra.run.smithi102.stdout:38654705687 2024-09-13T17:43:35.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.485+0000 7f2efdd1d640 1 -- 172.21.15.102:0/552244492 >> v1:172.21.15.102:6816/34937 conn(0x7f2ee4031160 legacy=0x7f2ee4033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.485+0000 7f2efdd1d640 1 -- 172.21.15.102:0/552244492 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ef8068920 msgr2=0x7f2ef811dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.485+0000 7f2efdd1d640 1 --2- 172.21.15.102:0/552244492 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ef8068920 0x7f2ef811dd80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2ee800eaa0 tx=0x7f2ee80090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.485+0000 7f2efdd1d640 1 -- 172.21.15.102:0/552244492 shutdown_connections 2024-09-13T17:43:35.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.485+0000 7f2efdd1d640 1 --2- 172.21.15.102:0/552244492 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ef812f310 0x7f2ef811e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.485+0000 7f2efdd1d640 1 --2- 172.21.15.102:0/552244492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ef809dc50 0x7f2ef811e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.485+0000 7f2efdd1d640 1 --2- 172.21.15.102:0/552244492 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ef8068920 0x7f2ef811dd80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.485+0000 7f2efdd1d640 1 -- 172.21.15.102:0/552244492 >> 172.21.15.102:0/552244492 conn(0x7f2ef806f320 msgr2=0x7f2ef809b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:35.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.485+0000 7f2efdd1d640 1 -- 172.21.15.102:0/552244492 shutdown_connections 2024-09-13T17:43:35.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.485+0000 7f2efdd1d640 1 -- 172.21.15.102:0/552244492 wait complete. 2024-09-13T17:43:35.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.485+0000 7f7dd4b1d640 1 -- 172.21.15.102:0/3964785251 >> v1:172.21.15.102:6816/34937 conn(0x7f7db0031160 legacy=0x7f7db0033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.485+0000 7f7dd4b1d640 1 -- 172.21.15.102:0/3964785251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dd006eb60 msgr2=0x7f7dd011c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.485+0000 7f7dd4b1d640 1 --2- 172.21.15.102:0/3964785251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dd006eb60 0x7f7dd011c260 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7db80018b0 tx=0x7f7db8007ef0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.485+0000 7f7dd4b1d640 1 -- 172.21.15.102:0/3964785251 shutdown_connections 2024-09-13T17:43:35.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.485+0000 7f7dd4b1d640 1 --2- 172.21.15.102:0/3964785251 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7dd009f7f0 0x7f7dd011c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.485+0000 7f7dd4b1d640 1 --2- 172.21.15.102:0/3964785251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dd006eb60 0x7f7dd011c260 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.485+0000 7f7dd4b1d640 1 --2- 172.21.15.102:0/3964785251 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7dd006d670 0x7f7dd011bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.485+0000 7f7dd4b1d640 1 -- 172.21.15.102:0/3964785251 >> 172.21.15.102:0/3964785251 conn(0x7f7dd0067bb0 msgr2=0x7f7dd012d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:35.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.485+0000 7f7dd4b1d640 1 -- 172.21.15.102:0/3964785251 shutdown_connections 2024-09-13T17:43:35.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.485+0000 7f7dd4b1d640 1 -- 172.21.15.102:0/3964785251 wait complete. 2024-09-13T17:43:35.498 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705686 got 38654705686 for osd.2 2024-09-13T17:43:35.499 DEBUG:teuthology.parallel:result is None 2024-09-13T17:43:35.499 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705687 got 38654705687 for osd.0 2024-09-13T17:43:35.499 DEBUG:teuthology.parallel:result is None 2024-09-13T17:43:35.499 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T17:43:35.500 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:43:35.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.630+0000 7ff864d9d640 1 Processor -- start 2024-09-13T17:43:35.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.630+0000 7ff864d9d640 1 -- start start 2024-09-13T17:43:35.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.630+0000 7ff864d9d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff86009abb0 0x7ff86012c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.630+0000 7ff864d9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff86012cae0 0x7ff860097490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.630+0000 7ff864d9d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff86006a530 con 0x7ff86012c6a0 2024-09-13T17:43:35.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.630+0000 7ff864d9d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff86009af80 con 0x7ff86009abb0 2024-09-13T17:43:35.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.630+0000 7ff864d9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff860098be0 con 0x7ff86012cae0 2024-09-13T17:43:35.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.631+0000 7ff85e59c640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff86009abb0 0x7ff86012c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.631+0000 7ff85ed9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff86012cae0 0x7ff860097490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.631+0000 7ff85e59c640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff86009abb0 0x7ff86012c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57586/0 (socket says 172.21.15.102:57586) 2024-09-13T17:43:35.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.631+0000 7ff85dd9b640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7ff86012c6a0 0x7ff860096d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:56266/0 (socket says 172.21.15.102:56266) 2024-09-13T17:43:35.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.631+0000 7ff85e59c640 1 -- 172.21.15.102:0/1349608816 learned_addr learned my addr 172.21.15.102:0/1349608816 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:35.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.631+0000 7ff85ed9d640 1 --2- 172.21.15.102:0/1349608816 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff86012cae0 0x7ff860097490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47348/0 (socket says 172.21.15.102:47348) 2024-09-13T17:43:35.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.631+0000 7ff85e59c640 1 -- 172.21.15.102:0/1349608816 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff86012cae0 msgr2=0x7ff860097490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:35.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.631+0000 7ff85e59c640 1 --2- 172.21.15.102:0/1349608816 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff86012cae0 0x7ff860097490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.631+0000 7ff85e59c640 1 -- 172.21.15.102:0/1349608816 >> v1:172.21.15.102:6789/0 conn(0x7ff86012c6a0 legacy=0x7ff860096d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.631+0000 7ff85e59c640 1 -- 172.21.15.102:0/1349608816 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff860098930 con 0x7ff86009abb0 2024-09-13T17:43:35.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.631+0000 7ff85d59a640 1 -- 172.21.15.102:0/1349608816 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2072451393 0 0) 0x7ff86006a530 con 0x7ff86012c6a0 2024-09-13T17:43:35.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.631+0000 7ff85ed9d640 1 --2- 172.21.15.102:0/1349608816 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff86012cae0 0x7ff860097490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:35.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.632+0000 7ff85e59c640 1 --2- 172.21.15.102:0/1349608816 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff86009abb0 0x7ff86012c160 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff850009910 tx=0x7ff8500091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97375b3289ad9ac0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:35.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.632+0000 7ff85d59a640 1 -- 172.21.15.102:0/1349608816 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85002ad20 con 0x7ff86009abb0 2024-09-13T17:43:35.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.632+0000 7ff85d59a640 1 -- 172.21.15.102:0/1349608816 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff85002ae80 con 0x7ff86009abb0 2024-09-13T17:43:35.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.632+0000 7ff85d59a640 1 -- 172.21.15.102:0/1349608816 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff850032670 con 0x7ff86009abb0 2024-09-13T17:43:35.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.632+0000 7ff864d9d640 1 -- 172.21.15.102:0/1349608816 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff86009abb0 msgr2=0x7ff86012c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.632+0000 7ff864d9d640 1 --2- 172.21.15.102:0/1349608816 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff86009abb0 0x7ff86012c160 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff850009910 tx=0x7ff8500091d0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.632+0000 7ff864d9d640 1 -- 172.21.15.102:0/1349608816 shutdown_connections 2024-09-13T17:43:35.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.632+0000 7ff864d9d640 1 --2- 172.21.15.102:0/1349608816 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff86012cae0 0x7ff860097490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.632+0000 7ff864d9d640 1 --2- 172.21.15.102:0/1349608816 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff86009abb0 0x7ff86012c160 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.632+0000 7ff864d9d640 1 -- 172.21.15.102:0/1349608816 >> 172.21.15.102:0/1349608816 conn(0x7ff86009e820 msgr2=0x7ff86009ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:35.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.632+0000 7ff864d9d640 1 -- 172.21.15.102:0/1349608816 shutdown_connections 2024-09-13T17:43:35.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.632+0000 7ff864d9d640 1 -- 172.21.15.102:0/1349608816 wait complete. 2024-09-13T17:43:35.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.633+0000 7ff864d9d640 1 Processor -- start 2024-09-13T17:43:35.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.633+0000 7ff864d9d640 1 -- start start 2024-09-13T17:43:35.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.633+0000 7ff864d9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff86009abb0 0x7ff86011b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.633+0000 7ff864d9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff86012c6a0 0x7ff860117d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.633+0000 7ff864d9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff86012cae0 0x7ff8601182a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.634+0000 7ff864d9d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86011aa40 con 0x7ff86012cae0 2024-09-13T17:43:35.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.634+0000 7ff864d9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8601190d0 con 0x7ff86012c6a0 2024-09-13T17:43:35.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.634+0000 7ff864d9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff86012a280 con 0x7ff86009abb0 2024-09-13T17:43:35.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.634+0000 7ff85e59c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff86009abb0 0x7ff86011b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.634+0000 7ff85ed9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff86012cae0 0x7ff8601182a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.634+0000 7ff85e59c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff86009abb0 0x7ff86011b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47354/0 (socket says 172.21.15.102:47354) 2024-09-13T17:43:35.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.634+0000 7ff85e59c640 1 -- 172.21.15.102:0/3591586529 learned_addr learned my addr 172.21.15.102:0/3591586529 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:35.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.634+0000 7ff85dd9b640 1 --2- 172.21.15.102:0/3591586529 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff86012c6a0 0x7ff860117d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.634+0000 7ff85ed9d640 1 -- 172.21.15.102:0/3591586529 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff86009abb0 msgr2=0x7ff86011b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.634+0000 7ff85ed9d640 1 --2- 172.21.15.102:0/3591586529 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff86009abb0 0x7ff86011b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.634+0000 7ff85ed9d640 1 -- 172.21.15.102:0/3591586529 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff86012c6a0 msgr2=0x7ff860117d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.634+0000 7ff85ed9d640 1 --2- 172.21.15.102:0/3591586529 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff86012c6a0 0x7ff860117d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.634+0000 7ff85ed9d640 1 -- 172.21.15.102:0/3591586529 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff858009a20 con 0x7ff86012cae0 2024-09-13T17:43:35.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.634+0000 7ff85e59c640 1 --2- 172.21.15.102:0/3591586529 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff86009abb0 0x7ff86011b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:35.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.634+0000 7ff85ed9d640 1 --2- 172.21.15.102:0/3591586529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff86012cae0 0x7ff8601182a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff858009f10 tx=0x7ff85800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:35.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.634+0000 7ff84e7fc640 1 -- 172.21.15.102:0/3591586529 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85800cdb0 con 0x7ff86012cae0 2024-09-13T17:43:35.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.634+0000 7ff84e7fc640 1 -- 172.21.15.102:0/3591586529 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff85800f040 con 0x7ff86012cae0 2024-09-13T17:43:35.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.634+0000 7ff84e7fc640 1 -- 172.21.15.102:0/3591586529 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8580146d0 con 0x7ff86012cae0 2024-09-13T17:43:35.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.634+0000 7ff864d9d640 1 -- 172.21.15.102:0/3591586529 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff850009450 con 0x7ff86012cae0 2024-09-13T17:43:35.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.634+0000 7ff864d9d640 1 -- 172.21.15.102:0/3591586529 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8601187e0 con 0x7ff86012cae0 2024-09-13T17:43:35.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.635+0000 7ff84e7fc640 1 -- 172.21.15.102:0/3591586529 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7ff858003020 con 0x7ff86012cae0 2024-09-13T17:43:35.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.635+0000 7ff84e7fc640 1 -- 172.21.15.102:0/3591586529 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7ff858042a50 con 0x7ff86012cae0 2024-09-13T17:43:35.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.635+0000 7ff864d9d640 1 -- 172.21.15.102:0/3591586529 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff860064000 con 0x7ff86012cae0 2024-09-13T17:43:35.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.639+0000 7ff84e7fc640 1 -- 172.21.15.102:0/3591586529 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7ff858018080 con 0x7ff86012cae0 2024-09-13T17:43:35.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.788+0000 7ff864d9d640 1 -- 172.21.15.102:0/3591586529 --> v1:172.21.15.102:6816/34937 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff860117b40 con 0x7ff840031190 2024-09-13T17:43:35.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.795+0000 7ff84e7fc640 1 -- 172.21.15.102:0/3591586529 <== mgr.4101 v1:172.21.15.102:6816/34937 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+276350 (unknown 2965378022 0 280590427) 0x7ff850032da0 con 0x7ff840031190 2024-09-13T17:43:35.799 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:43:35.801 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-09-13T17:43:35.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.798+0000 7ff864d9d640 1 -- 172.21.15.102:0/3591586529 >> v1:172.21.15.102:6816/34937 conn(0x7ff840031190 legacy=0x7ff840033640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.798+0000 7ff864d9d640 1 -- 172.21.15.102:0/3591586529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff86012cae0 msgr2=0x7ff8601182a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.798+0000 7ff864d9d640 1 --2- 172.21.15.102:0/3591586529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff86012cae0 0x7ff8601182a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff858009f10 tx=0x7ff85800b040 comp rx=0 tx=0).stop 2024-09-13T17:43:35.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.798+0000 7ff864d9d640 1 -- 172.21.15.102:0/3591586529 shutdown_connections 2024-09-13T17:43:35.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.798+0000 7ff864d9d640 1 --2- 172.21.15.102:0/3591586529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff86012cae0 0x7ff8601182a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.798+0000 7ff864d9d640 1 --2- 172.21.15.102:0/3591586529 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff86012c6a0 0x7ff860117d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.798+0000 7ff864d9d640 1 --2- 172.21.15.102:0/3591586529 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff86009abb0 0x7ff86011b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.798+0000 7ff864d9d640 1 -- 172.21.15.102:0/3591586529 >> 172.21.15.102:0/3591586529 conn(0x7ff86009e820 msgr2=0x7ff86009b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:35.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.798+0000 7ff864d9d640 1 -- 172.21.15.102:0/3591586529 shutdown_connections 2024-09-13T17:43:35.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.798+0000 7ff864d9d640 1 -- 172.21.15.102:0/3591586529 wait complete. 2024-09-13T17:43:35.810 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2024-09-13T17:43:34.433558+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594114,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":130,"num_write_kb":1125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":98,"ondisk_log_size":98,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1191432,"kb_used_data":2000,"kb_used_omap":6,"kb_used_meta":1189369,"kb_avail":376295928,"statfs":{"total":386547056640,"available":385327030272,"internally_reserved":0,"allocated":2048000,"data_stored":1460124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6816,"internal_metadata":1217914208},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003250"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.395985+0000","last_change":"2024-09-13T17:42:07.332154+0000","last_active":"2024-09-13T17:42:47.395985+0000","last_peered":"2024-09-13T17:42:47.395985+0000","last_clean":"2024-09-13T17:42:47.395985+0000","last_became_active":"2024-09-13T17:42:07.331874+0000","last_became_peered":"2024-09-13T17:42:07.331874+0000","last_unstale":"2024-09-13T17:42:47.395985+0000","last_undegraded":"2024-09-13T17:42:47.395985+0000","last_fullsized":"2024-09-13T17:42:47.395985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:10:22.943375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.396057+0000","last_change":"2024-09-13T17:42:07.326086+0000","last_active":"2024-09-13T17:42:47.396057+0000","last_peered":"2024-09-13T17:42:47.396057+0000","last_clean":"2024-09-13T17:42:47.396057+0000","last_became_active":"2024-09-13T17:42:07.325830+0000","last_became_peered":"2024-09-13T17:42:07.325830+0000","last_unstale":"2024-09-13T17:42:47.396057+0000","last_undegraded":"2024-09-13T17:42:47.396057+0000","last_fullsized":"2024-09-13T17:42:47.396057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:54.719512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.396454+0000","last_change":"2024-09-13T17:42:07.326953+0000","last_active":"2024-09-13T17:42:47.396454+0000","last_peered":"2024-09-13T17:42:47.396454+0000","last_clean":"2024-09-13T17:42:47.396454+0000","last_became_active":"2024-09-13T17:42:07.326717+0000","last_became_peered":"2024-09-13T17:42:07.326717+0000","last_unstale":"2024-09-13T17:42:47.396454+0000","last_undegraded":"2024-09-13T17:42:47.396454+0000","last_fullsized":"2024-09-13T17:42:47.396454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:20.647404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.396025+0000","last_change":"2024-09-13T17:42:07.326356+0000","last_active":"2024-09-13T17:42:47.396025+0000","last_peered":"2024-09-13T17:42:47.396025+0000","last_clean":"2024-09-13T17:42:47.396025+0000","last_became_active":"2024-09-13T17:42:07.326127+0000","last_became_peered":"2024-09-13T17:42:07.326127+0000","last_unstale":"2024-09-13T17:42:47.396025+0000","last_undegraded":"2024-09-13T17:42:47.396025+0000","last_fullsized":"2024-09-13T17:42:47.396025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:44.405426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.396070+0000","last_change":"2024-09-13T17:42:07.330092+0000","last_active":"2024-09-13T17:42:47.396070+0000","last_peered":"2024-09-13T17:42:47.396070+0000","last_clean":"2024-09-13T17:42:47.396070+0000","last_became_active":"2024-09-13T17:42:07.327658+0000","last_became_peered":"2024-09-13T17:42:07.327658+0000","last_unstale":"2024-09-13T17:42:47.396070+0000","last_undegraded":"2024-09-13T17:42:47.396070+0000","last_fullsized":"2024-09-13T17:42:47.396070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:24:02.920430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:48.399934+0000","last_change":"2024-09-13T17:42:48.399934+0000","last_active":"2024-09-13T17:42:48.399934+0000","last_peered":"2024-09-13T17:42:48.399934+0000","last_clean":"2024-09-13T17:42:48.399934+0000","last_became_active":"2024-09-13T17:42:48.399527+0000","last_became_peered":"2024-09-13T17:42:48.399527+0000","last_unstale":"2024-09-13T17:42:48.399934+0000","last_undegraded":"2024-09-13T17:42:48.399934+0000","last_fullsized":"2024-09-13T17:42:48.399934+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:03:44.489038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.389584+0000","last_change":"2024-09-13T17:42:07.307630+0000","last_active":"2024-09-13T17:42:47.389584+0000","last_peered":"2024-09-13T17:42:47.389584+0000","last_clean":"2024-09-13T17:42:47.389584+0000","last_became_active":"2024-09-13T17:42:07.307382+0000","last_became_peered":"2024-09-13T17:42:07.307382+0000","last_unstale":"2024-09-13T17:42:47.389584+0000","last_undegraded":"2024-09-13T17:42:47.389584+0000","last_fullsized":"2024-09-13T17:42:47.389584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:26:32.846287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398633+0000","last_change":"2024-09-13T17:42:07.320736+0000","last_active":"2024-09-13T17:42:47.398633+0000","last_peered":"2024-09-13T17:42:47.398633+0000","last_clean":"2024-09-13T17:42:47.398633+0000","last_became_active":"2024-09-13T17:42:07.319801+0000","last_became_peered":"2024-09-13T17:42:07.319801+0000","last_unstale":"2024-09-13T17:42:47.398633+0000","last_undegraded":"2024-09-13T17:42:47.398633+0000","last_fullsized":"2024-09-13T17:42:47.398633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:28:32.030397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398290+0000","last_change":"2024-09-13T17:42:07.335208+0000","last_active":"2024-09-13T17:42:47.398290+0000","last_peered":"2024-09-13T17:42:47.398290+0000","last_clean":"2024-09-13T17:42:47.398290+0000","last_became_active":"2024-09-13T17:42:07.334956+0000","last_became_peered":"2024-09-13T17:42:07.334956+0000","last_unstale":"2024-09-13T17:42:47.398290+0000","last_undegraded":"2024-09-13T17:42:47.398290+0000","last_fullsized":"2024-09-13T17:42:47.398290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:59:53.544086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.397884+0000","last_change":"2024-09-13T17:42:07.317057+0000","last_active":"2024-09-13T17:42:47.397884+0000","last_peered":"2024-09-13T17:42:47.397884+0000","last_clean":"2024-09-13T17:42:47.397884+0000","last_became_active":"2024-09-13T17:42:07.316429+0000","last_became_peered":"2024-09-13T17:42:07.316429+0000","last_unstale":"2024-09-13T17:42:47.397884+0000","last_undegraded":"2024-09-13T17:42:47.397884+0000","last_fullsized":"2024-09-13T17:42:47.397884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:59:53.544086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.397688+0000","last_change":"2024-09-13T17:42:07.321106+0000","last_active":"2024-09-13T17:42:47.397688+0000","last_peered":"2024-09-13T17:42:47.397688+0000","last_clean":"2024-09-13T17:42:47.397688+0000","last_became_active":"2024-09-13T17:42:07.320827+0000","last_became_peered":"2024-09-13T17:42:07.320827+0000","last_unstale":"2024-09-13T17:42:47.397688+0000","last_undegraded":"2024-09-13T17:42:47.397688+0000","last_fullsized":"2024-09-13T17:42:47.397688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:34:34.794230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.395720+0000","last_change":"2024-09-13T17:42:07.331865+0000","last_active":"2024-09-13T17:42:47.395720+0000","last_peered":"2024-09-13T17:42:47.395720+0000","last_clean":"2024-09-13T17:42:47.395720+0000","last_became_active":"2024-09-13T17:42:07.331419+0000","last_became_peered":"2024-09-13T17:42:07.331419+0000","last_unstale":"2024-09-13T17:42:47.395720+0000","last_undegraded":"2024-09-13T17:42:47.395720+0000","last_fullsized":"2024-09-13T17:42:47.395720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:48.155823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:48.400031+0000","last_change":"2024-09-13T17:42:48.400031+0000","last_active":"2024-09-13T17:42:48.400031+0000","last_peered":"2024-09-13T17:42:48.400031+0000","last_clean":"2024-09-13T17:42:48.400031+0000","last_became_active":"2024-09-13T17:42:48.399662+0000","last_became_peered":"2024-09-13T17:42:48.399662+0000","last_unstale":"2024-09-13T17:42:48.400031+0000","last_undegraded":"2024-09-13T17:42:48.400031+0000","last_fullsized":"2024-09-13T17:42:48.400031+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:09.715214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.397259+0000","last_change":"2024-09-13T17:42:07.324771+0000","last_active":"2024-09-13T17:42:47.397259+0000","last_peered":"2024-09-13T17:42:47.397259+0000","last_clean":"2024-09-13T17:42:47.397259+0000","last_became_active":"2024-09-13T17:42:07.323820+0000","last_became_peered":"2024-09-13T17:42:07.323820+0000","last_unstale":"2024-09-13T17:42:47.397259+0000","last_undegraded":"2024-09-13T17:42:47.397259+0000","last_fullsized":"2024-09-13T17:42:47.397259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:24:02.920430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.395285+0000","last_change":"2024-09-13T17:42:07.325827+0000","last_active":"2024-09-13T17:42:47.395285+0000","last_peered":"2024-09-13T17:42:47.395285+0000","last_clean":"2024-09-13T17:42:47.395285+0000","last_became_active":"2024-09-13T17:42:07.325589+0000","last_became_peered":"2024-09-13T17:42:07.325589+0000","last_unstale":"2024-09-13T17:42:47.395285+0000","last_undegraded":"2024-09-13T17:42:47.395285+0000","last_fullsized":"2024-09-13T17:42:47.395285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:43.466051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.388642+0000","last_change":"2024-09-13T17:42:07.315478+0000","last_active":"2024-09-13T17:42:47.388642+0000","last_peered":"2024-09-13T17:42:47.388642+0000","last_clean":"2024-09-13T17:42:47.388642+0000","last_became_active":"2024-09-13T17:42:07.315218+0000","last_became_peered":"2024-09-13T17:42:07.315218+0000","last_unstale":"2024-09-13T17:42:47.388642+0000","last_undegraded":"2024-09-13T17:42:47.388642+0000","last_fullsized":"2024-09-13T17:42:47.388642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:06:44.563276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398180+0000","last_change":"2024-09-13T17:42:07.328192+0000","last_active":"2024-09-13T17:42:47.398180+0000","last_peered":"2024-09-13T17:42:47.398180+0000","last_clean":"2024-09-13T17:42:47.398180+0000","last_became_active":"2024-09-13T17:42:07.327951+0000","last_became_peered":"2024-09-13T17:42:07.327951+0000","last_unstale":"2024-09-13T17:42:47.398180+0000","last_undegraded":"2024-09-13T17:42:47.398180+0000","last_fullsized":"2024-09-13T17:42:47.398180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:28:32.030397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.396085+0000","last_change":"2024-09-13T17:42:07.314816+0000","last_active":"2024-09-13T17:42:47.396085+0000","last_peered":"2024-09-13T17:42:47.396085+0000","last_clean":"2024-09-13T17:42:47.396085+0000","last_became_active":"2024-09-13T17:42:07.314464+0000","last_became_peered":"2024-09-13T17:42:07.314464+0000","last_unstale":"2024-09-13T17:42:47.396085+0000","last_undegraded":"2024-09-13T17:42:47.396085+0000","last_fullsized":"2024-09-13T17:42:47.396085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:17.483066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.395672+0000","last_change":"2024-09-13T17:42:07.332269+0000","last_active":"2024-09-13T17:42:47.395672+0000","last_peered":"2024-09-13T17:42:47.395672+0000","last_clean":"2024-09-13T17:42:47.395672+0000","last_became_active":"2024-09-13T17:42:07.332040+0000","last_became_peered":"2024-09-13T17:42:07.332040+0000","last_unstale":"2024-09-13T17:42:47.395672+0000","last_undegraded":"2024-09-13T17:42:47.395672+0000","last_fullsized":"2024-09-13T17:42:47.395672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:48:49.368747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398189+0000","last_change":"2024-09-13T17:42:07.335425+0000","last_active":"2024-09-13T17:42:47.398189+0000","last_peered":"2024-09-13T17:42:47.398189+0000","last_clean":"2024-09-13T17:42:47.398189+0000","last_became_active":"2024-09-13T17:42:07.335173+0000","last_became_peered":"2024-09-13T17:42:07.335173+0000","last_unstale":"2024-09-13T17:42:47.398189+0000","last_undegraded":"2024-09-13T17:42:47.398189+0000","last_fullsized":"2024-09-13T17:42:47.398189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:00:10.355609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.397591+0000","last_change":"2024-09-13T17:42:07.331183+0000","last_active":"2024-09-13T17:42:47.397591+0000","last_peered":"2024-09-13T17:42:47.397591+0000","last_clean":"2024-09-13T17:42:47.397591+0000","last_became_active":"2024-09-13T17:42:07.330936+0000","last_became_peered":"2024-09-13T17:42:07.330936+0000","last_unstale":"2024-09-13T17:42:47.397591+0000","last_undegraded":"2024-09-13T17:42:47.397591+0000","last_fullsized":"2024-09-13T17:42:47.397591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:42:38.505262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.396192+0000","last_change":"2024-09-13T17:42:07.325877+0000","last_active":"2024-09-13T17:42:47.396192+0000","last_peered":"2024-09-13T17:42:47.396192+0000","last_clean":"2024-09-13T17:42:47.396192+0000","last_became_active":"2024-09-13T17:42:07.325657+0000","last_became_peered":"2024-09-13T17:42:07.325657+0000","last_unstale":"2024-09-13T17:42:47.396192+0000","last_undegraded":"2024-09-13T17:42:47.396192+0000","last_fullsized":"2024-09-13T17:42:47.396192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:48.663311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.389614+0000","last_change":"2024-09-13T17:42:07.322059+0000","last_active":"2024-09-13T17:42:47.389614+0000","last_peered":"2024-09-13T17:42:47.389614+0000","last_clean":"2024-09-13T17:42:47.389614+0000","last_became_active":"2024-09-13T17:42:07.312536+0000","last_became_peered":"2024-09-13T17:42:07.312536+0000","last_unstale":"2024-09-13T17:42:47.389614+0000","last_undegraded":"2024-09-13T17:42:47.389614+0000","last_fullsized":"2024-09-13T17:42:47.389614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:14:08.780968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398684+0000","last_change":"2024-09-13T17:42:07.336557+0000","last_active":"2024-09-13T17:42:47.398684+0000","last_peered":"2024-09-13T17:42:47.398684+0000","last_clean":"2024-09-13T17:42:47.398684+0000","last_became_active":"2024-09-13T17:42:07.335820+0000","last_became_peered":"2024-09-13T17:42:07.335820+0000","last_unstale":"2024-09-13T17:42:47.398684+0000","last_undegraded":"2024-09-13T17:42:47.398684+0000","last_fullsized":"2024-09-13T17:42:47.398684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:20.647404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398429+0000","last_change":"2024-09-13T17:42:07.337695+0000","last_active":"2024-09-13T17:42:47.398429+0000","last_peered":"2024-09-13T17:42:47.398429+0000","last_clean":"2024-09-13T17:42:47.398429+0000","last_became_active":"2024-09-13T17:42:07.337419+0000","last_became_peered":"2024-09-13T17:42:07.337419+0000","last_unstale":"2024-09-13T17:42:47.398429+0000","last_undegraded":"2024-09-13T17:42:47.398429+0000","last_fullsized":"2024-09-13T17:42:47.398429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:22:40.880345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.397901+0000","last_change":"2024-09-13T17:42:07.315008+0000","last_active":"2024-09-13T17:42:47.397901+0000","last_peered":"2024-09-13T17:42:47.397901+0000","last_clean":"2024-09-13T17:42:47.397901+0000","last_became_active":"2024-09-13T17:42:07.313477+0000","last_became_peered":"2024-09-13T17:42:07.313477+0000","last_unstale":"2024-09-13T17:42:47.397901+0000","last_undegraded":"2024-09-13T17:42:47.397901+0000","last_fullsized":"2024-09-13T17:42:47.397901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:25.632002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:48.397637+0000","last_change":"2024-09-13T17:42:48.397637+0000","last_active":"2024-09-13T17:42:48.397637+0000","last_peered":"2024-09-13T17:42:48.397637+0000","last_clean":"2024-09-13T17:42:48.397637+0000","last_became_active":"2024-09-13T17:42:48.397311+0000","last_became_peered":"2024-09-13T17:42:48.397311+0000","last_unstale":"2024-09-13T17:42:48.397637+0000","last_undegraded":"2024-09-13T17:42:48.397637+0000","last_fullsized":"2024-09-13T17:42:48.397637+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:12:22.089693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398470+0000","last_change":"2024-09-13T17:42:07.313528+0000","last_active":"2024-09-13T17:42:47.398470+0000","last_peered":"2024-09-13T17:42:47.398470+0000","last_clean":"2024-09-13T17:42:47.398470+0000","last_became_active":"2024-09-13T17:42:07.313092+0000","last_became_peered":"2024-09-13T17:42:07.313092+0000","last_unstale":"2024-09-13T17:42:47.398470+0000","last_undegraded":"2024-09-13T17:42:47.398470+0000","last_fullsized":"2024-09-13T17:42:47.398470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:48:49.368747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.390660+0000","last_change":"2024-09-13T17:42:07.314816+0000","last_active":"2024-09-13T17:42:47.390660+0000","last_peered":"2024-09-13T17:42:47.390660+0000","last_clean":"2024-09-13T17:42:47.390660+0000","last_became_active":"2024-09-13T17:42:07.314574+0000","last_became_peered":"2024-09-13T17:42:07.314574+0000","last_unstale":"2024-09-13T17:42:47.390660+0000","last_undegraded":"2024-09-13T17:42:47.390660+0000","last_fullsized":"2024-09-13T17:42:47.390660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:03:35.976183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.396358+0000","last_change":"2024-09-13T17:42:07.328522+0000","last_active":"2024-09-13T17:42:47.396358+0000","last_peered":"2024-09-13T17:42:47.396358+0000","last_clean":"2024-09-13T17:42:47.396358+0000","last_became_active":"2024-09-13T17:42:07.327893+0000","last_became_peered":"2024-09-13T17:42:07.327893+0000","last_unstale":"2024-09-13T17:42:47.396358+0000","last_undegraded":"2024-09-13T17:42:47.396358+0000","last_fullsized":"2024-09-13T17:42:47.396358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:02:50.060149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.389675+0000","last_change":"2024-09-13T17:42:07.321900+0000","last_active":"2024-09-13T17:42:47.389675+0000","last_peered":"2024-09-13T17:42:47.389675+0000","last_clean":"2024-09-13T17:42:47.389675+0000","last_became_active":"2024-09-13T17:42:07.319230+0000","last_became_peered":"2024-09-13T17:42:07.319230+0000","last_unstale":"2024-09-13T17:42:47.389675+0000","last_undegraded":"2024-09-13T17:42:47.389675+0000","last_fullsized":"2024-09-13T17:42:47.389675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:50:46.442172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398698+0000","last_change":"2024-09-13T17:42:07.315846+0000","last_active":"2024-09-13T17:42:47.398698+0000","last_peered":"2024-09-13T17:42:47.398698+0000","last_clean":"2024-09-13T17:42:47.398698+0000","last_became_active":"2024-09-13T17:42:07.315513+0000","last_became_peered":"2024-09-13T17:42:07.315513+0000","last_unstale":"2024-09-13T17:42:47.398698+0000","last_undegraded":"2024-09-13T17:42:47.398698+0000","last_fullsized":"2024-09-13T17:42:47.398698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:56:21.343829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:48.515524+0000","last_change":"2024-09-13T17:42:48.515524+0000","last_active":"2024-09-13T17:42:48.515524+0000","last_peered":"2024-09-13T17:42:48.515524+0000","last_clean":"2024-09-13T17:42:48.515524+0000","last_became_active":"2024-09-13T17:42:48.515203+0000","last_became_peered":"2024-09-13T17:42:48.515203+0000","last_unstale":"2024-09-13T17:42:48.515524+0000","last_undegraded":"2024-09-13T17:42:48.515524+0000","last_fullsized":"2024-09-13T17:42:48.515524+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:43:52.887484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.390769+0000","last_change":"2024-09-13T17:42:06.308504+0000","last_active":"2024-09-13T17:42:47.390769+0000","last_peered":"2024-09-13T17:42:47.390769+0000","last_clean":"2024-09-13T17:42:47.390769+0000","last_became_active":"2024-09-13T17:42:06.308199+0000","last_became_peered":"2024-09-13T17:42:06.308199+0000","last_unstale":"2024-09-13T17:42:47.390769+0000","last_undegraded":"2024-09-13T17:42:47.390769+0000","last_fullsized":"2024-09-13T17:42:47.390769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:05.930342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.390353+0000","last_change":"2024-09-13T17:42:07.322597+0000","last_active":"2024-09-13T17:42:47.390353+0000","last_peered":"2024-09-13T17:42:47.390353+0000","last_clean":"2024-09-13T17:42:47.390353+0000","last_became_active":"2024-09-13T17:42:07.321945+0000","last_became_peered":"2024-09-13T17:42:07.321945+0000","last_unstale":"2024-09-13T17:42:47.390353+0000","last_undegraded":"2024-09-13T17:42:47.390353+0000","last_fullsized":"2024-09-13T17:42:47.390353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:03:28.132876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.396318+0000","last_change":"2024-09-13T17:42:06.306367+0000","last_active":"2024-09-13T17:42:47.396318+0000","last_peered":"2024-09-13T17:42:47.396318+0000","last_clean":"2024-09-13T17:42:47.396318+0000","last_became_active":"2024-09-13T17:42:06.306146+0000","last_became_peered":"2024-09-13T17:42:06.306146+0000","last_unstale":"2024-09-13T17:42:47.396318+0000","last_undegraded":"2024-09-13T17:42:47.396318+0000","last_fullsized":"2024-09-13T17:42:47.396318+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:45:01.750995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.396643+0000","last_change":"2024-09-13T17:42:07.329528+0000","last_active":"2024-09-13T17:42:47.396643+0000","last_peered":"2024-09-13T17:42:47.396643+0000","last_clean":"2024-09-13T17:42:47.396643+0000","last_became_active":"2024-09-13T17:42:07.329296+0000","last_became_peered":"2024-09-13T17:42:07.329296+0000","last_unstale":"2024-09-13T17:42:47.396643+0000","last_undegraded":"2024-09-13T17:42:47.396643+0000","last_fullsized":"2024-09-13T17:42:47.396643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:33:57.543704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.389827+0000","last_change":"2024-09-13T17:42:06.312798+0000","last_active":"2024-09-13T17:42:47.389827+0000","last_peered":"2024-09-13T17:42:47.389827+0000","last_clean":"2024-09-13T17:42:47.389827+0000","last_became_active":"2024-09-13T17:42:06.312388+0000","last_became_peered":"2024-09-13T17:42:06.312388+0000","last_unstale":"2024-09-13T17:42:47.389827+0000","last_undegraded":"2024-09-13T17:42:47.389827+0000","last_fullsized":"2024-09-13T17:42:47.389827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:56:20.339850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:48.514881+0000","last_change":"2024-09-13T17:42:48.514881+0000","last_active":"2024-09-13T17:42:48.514881+0000","last_peered":"2024-09-13T17:42:48.514881+0000","last_clean":"2024-09-13T17:42:48.514881+0000","last_became_active":"2024-09-13T17:42:48.514539+0000","last_became_peered":"2024-09-13T17:42:48.514539+0000","last_unstale":"2024-09-13T17:42:48.514881+0000","last_undegraded":"2024-09-13T17:42:48.514881+0000","last_fullsized":"2024-09-13T17:42:48.514881+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:14:13.836498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398776+0000","last_change":"2024-09-13T17:42:06.305806+0000","last_active":"2024-09-13T17:42:47.398776+0000","last_peered":"2024-09-13T17:42:47.398776+0000","last_clean":"2024-09-13T17:42:47.398776+0000","last_became_active":"2024-09-13T17:42:06.305576+0000","last_became_peered":"2024-09-13T17:42:06.305576+0000","last_unstale":"2024-09-13T17:42:47.398776+0000","last_undegraded":"2024-09-13T17:42:47.398776+0000","last_fullsized":"2024-09-13T17:42:47.398776+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:47:16.711770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398428+0000","last_change":"2024-09-13T17:42:06.304106+0000","last_active":"2024-09-13T17:42:47.398428+0000","last_peered":"2024-09-13T17:42:47.398428+0000","last_clean":"2024-09-13T17:42:47.398428+0000","last_became_active":"2024-09-13T17:42:06.303845+0000","last_became_peered":"2024-09-13T17:42:06.303845+0000","last_unstale":"2024-09-13T17:42:47.398428+0000","last_undegraded":"2024-09-13T17:42:47.398428+0000","last_fullsized":"2024-09-13T17:42:47.398428+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:36:02.512243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.388874+0000","last_change":"2024-09-13T17:42:07.331147+0000","last_active":"2024-09-13T17:42:47.388874+0000","last_peered":"2024-09-13T17:42:47.388874+0000","last_clean":"2024-09-13T17:42:47.388874+0000","last_became_active":"2024-09-13T17:42:07.329386+0000","last_became_peered":"2024-09-13T17:42:07.329386+0000","last_unstale":"2024-09-13T17:42:47.388874+0000","last_undegraded":"2024-09-13T17:42:47.388874+0000","last_fullsized":"2024-09-13T17:42:47.388874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:59:21.210970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.395917+0000","last_change":"2024-09-13T17:42:06.304872+0000","last_active":"2024-09-13T17:42:47.395917+0000","last_peered":"2024-09-13T17:42:47.395917+0000","last_clean":"2024-09-13T17:42:47.395917+0000","last_became_active":"2024-09-13T17:42:06.304639+0000","last_became_peered":"2024-09-13T17:42:06.304639+0000","last_unstale":"2024-09-13T17:42:47.395917+0000","last_undegraded":"2024-09-13T17:42:47.395917+0000","last_fullsized":"2024-09-13T17:42:47.395917+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:40.147410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.397065+0000","last_change":"2024-09-13T17:42:07.331235+0000","last_active":"2024-09-13T17:42:47.397065+0000","last_peered":"2024-09-13T17:42:47.397065+0000","last_clean":"2024-09-13T17:42:47.397065+0000","last_became_active":"2024-09-13T17:42:07.330822+0000","last_became_peered":"2024-09-13T17:42:07.330822+0000","last_unstale":"2024-09-13T17:42:47.397065+0000","last_undegraded":"2024-09-13T17:42:47.397065+0000","last_fullsized":"2024-09-13T17:42:47.397065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:38:04.013092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.390635+0000","last_change":"2024-09-13T17:42:06.310319+0000","last_active":"2024-09-13T17:42:47.390635+0000","last_peered":"2024-09-13T17:42:47.390635+0000","last_clean":"2024-09-13T17:42:47.390635+0000","last_became_active":"2024-09-13T17:42:06.310085+0000","last_became_peered":"2024-09-13T17:42:06.310085+0000","last_unstale":"2024-09-13T17:42:47.390635+0000","last_undegraded":"2024-09-13T17:42:47.390635+0000","last_fullsized":"2024-09-13T17:42:47.390635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:53.715533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.395288+0000","last_change":"2024-09-13T17:42:07.333488+0000","last_active":"2024-09-13T17:42:47.395288+0000","last_peered":"2024-09-13T17:42:47.395288+0000","last_clean":"2024-09-13T17:42:47.395288+0000","last_became_active":"2024-09-13T17:42:07.332200+0000","last_became_peered":"2024-09-13T17:42:07.332200+0000","last_unstale":"2024-09-13T17:42:47.395288+0000","last_undegraded":"2024-09-13T17:42:47.395288+0000","last_fullsized":"2024-09-13T17:42:47.395288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:52.666716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.396110+0000","last_change":"2024-09-13T17:42:06.308854+0000","last_active":"2024-09-13T17:42:47.396110+0000","last_peered":"2024-09-13T17:42:47.396110+0000","last_clean":"2024-09-13T17:42:47.396110+0000","last_became_active":"2024-09-13T17:42:06.307173+0000","last_became_peered":"2024-09-13T17:42:06.307173+0000","last_unstale":"2024-09-13T17:42:47.396110+0000","last_undegraded":"2024-09-13T17:42:47.396110+0000","last_fullsized":"2024-09-13T17:42:47.396110+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:59.621392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.395858+0000","last_change":"2024-09-13T17:42:07.327431+0000","last_active":"2024-09-13T17:42:47.395858+0000","last_peered":"2024-09-13T17:42:47.395858+0000","last_clean":"2024-09-13T17:42:47.395858+0000","last_became_active":"2024-09-13T17:42:07.327202+0000","last_became_peered":"2024-09-13T17:42:07.327202+0000","last_unstale":"2024-09-13T17:42:47.395858+0000","last_undegraded":"2024-09-13T17:42:47.395858+0000","last_fullsized":"2024-09-13T17:42:47.395858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:42:38.505262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.390949+0000","last_change":"2024-09-13T17:42:06.308795+0000","last_active":"2024-09-13T17:42:47.390949+0000","last_peered":"2024-09-13T17:42:47.390949+0000","last_clean":"2024-09-13T17:42:47.390949+0000","last_became_active":"2024-09-13T17:42:06.308557+0000","last_became_peered":"2024-09-13T17:42:06.308557+0000","last_unstale":"2024-09-13T17:42:47.390949+0000","last_undegraded":"2024-09-13T17:42:47.390949+0000","last_fullsized":"2024-09-13T17:42:47.390949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:51.662738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398005+0000","last_change":"2024-09-13T17:42:07.327849+0000","last_active":"2024-09-13T17:42:47.398005+0000","last_peered":"2024-09-13T17:42:47.398005+0000","last_clean":"2024-09-13T17:42:47.398005+0000","last_became_active":"2024-09-13T17:42:07.327578+0000","last_became_peered":"2024-09-13T17:42:07.327578+0000","last_unstale":"2024-09-13T17:42:47.398005+0000","last_undegraded":"2024-09-13T17:42:47.398005+0000","last_fullsized":"2024-09-13T17:42:47.398005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:34:34.794230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.390194+0000","last_change":"2024-09-13T17:42:06.310708+0000","last_active":"2024-09-13T17:42:47.390194+0000","last_peered":"2024-09-13T17:42:47.390194+0000","last_clean":"2024-09-13T17:42:47.390194+0000","last_became_active":"2024-09-13T17:42:06.310476+0000","last_became_peered":"2024-09-13T17:42:06.310476+0000","last_unstale":"2024-09-13T17:42:47.390194+0000","last_undegraded":"2024-09-13T17:42:47.390194+0000","last_fullsized":"2024-09-13T17:42:47.390194+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:10:21.939396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.390317+0000","last_change":"2024-09-13T17:42:07.317665+0000","last_active":"2024-09-13T17:42:47.390317+0000","last_peered":"2024-09-13T17:42:47.390317+0000","last_clean":"2024-09-13T17:42:47.390317+0000","last_became_active":"2024-09-13T17:42:07.317401+0000","last_became_peered":"2024-09-13T17:42:07.317401+0000","last_unstale":"2024-09-13T17:42:47.390317+0000","last_undegraded":"2024-09-13T17:42:47.390317+0000","last_fullsized":"2024-09-13T17:42:47.390317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:12:55.460668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.390117+0000","last_change":"2024-09-13T17:42:06.313488+0000","last_active":"2024-09-13T17:42:47.390117+0000","last_peered":"2024-09-13T17:42:47.390117+0000","last_clean":"2024-09-13T17:42:47.390117+0000","last_became_active":"2024-09-13T17:42:06.313241+0000","last_became_peered":"2024-09-13T17:42:06.313241+0000","last_unstale":"2024-09-13T17:42:47.390117+0000","last_undegraded":"2024-09-13T17:42:47.390117+0000","last_fullsized":"2024-09-13T17:42:47.390117+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:48:48.364768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.395702+0000","last_change":"2024-09-13T17:42:07.333605+0000","last_active":"2024-09-13T17:42:47.395702+0000","last_peered":"2024-09-13T17:42:47.395702+0000","last_clean":"2024-09-13T17:42:47.395702+0000","last_became_active":"2024-09-13T17:42:07.332509+0000","last_became_peered":"2024-09-13T17:42:07.332509+0000","last_unstale":"2024-09-13T17:42:47.395702+0000","last_undegraded":"2024-09-13T17:42:47.395702+0000","last_fullsized":"2024-09-13T17:42:47.395702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:26:11.888585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.389821+0000","last_change":"2024-09-13T17:42:06.310629+0000","last_active":"2024-09-13T17:42:47.389821+0000","last_peered":"2024-09-13T17:42:47.389821+0000","last_clean":"2024-09-13T17:42:47.389821+0000","last_became_active":"2024-09-13T17:42:06.310103+0000","last_became_peered":"2024-09-13T17:42:06.310103+0000","last_unstale":"2024-09-13T17:42:47.389821+0000","last_undegraded":"2024-09-13T17:42:47.389821+0000","last_fullsized":"2024-09-13T17:42:47.389821+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:42.121791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398502+0000","last_change":"2024-09-13T17:42:07.335957+0000","last_active":"2024-09-13T17:42:47.398502+0000","last_peered":"2024-09-13T17:42:47.398502+0000","last_clean":"2024-09-13T17:42:47.398502+0000","last_became_active":"2024-09-13T17:42:07.331187+0000","last_became_peered":"2024-09-13T17:42:07.331187+0000","last_unstale":"2024-09-13T17:42:47.398502+0000","last_undegraded":"2024-09-13T17:42:47.398502+0000","last_fullsized":"2024-09-13T17:42:47.398502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:44.405426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":87,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398917+0000","last_change":"2024-09-13T17:41:55.654539+0000","last_active":"2024-09-13T17:42:47.398917+0000","last_peered":"2024-09-13T17:42:47.398917+0000","last_clean":"2024-09-13T17:42:47.398917+0000","last_became_active":"2024-09-13T17:41:55.653536+0000","last_became_peered":"2024-09-13T17:41:55.653536+0000","last_unstale":"2024-09-13T17:42:47.398917+0000","last_undegraded":"2024-09-13T17:42:47.398917+0000","last_fullsized":"2024-09-13T17:42:47.398917+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:54.532965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:54.532965+0000","last_clean_scrub_stamp":"2024-09-13T17:41:54.532965+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:56.662588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.391053+0000","last_change":"2024-09-13T17:42:02.137356+0000","last_active":"2024-09-13T17:42:47.391053+0000","last_peered":"2024-09-13T17:42:47.391053+0000","last_clean":"2024-09-13T17:42:47.391053+0000","last_became_active":"2024-09-13T17:42:00.116534+0000","last_became_peered":"2024-09-13T17:42:00.116534+0000","last_unstale":"2024-09-13T17:42:47.391053+0000","last_undegraded":"2024-09-13T17:42:47.391053+0000","last_fullsized":"2024-09-13T17:42:47.391053+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:06.665067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000321799,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.390368+0000","last_change":"2024-09-13T17:42:06.312144+0000","last_active":"2024-09-13T17:42:47.390368+0000","last_peered":"2024-09-13T17:42:47.390368+0000","last_clean":"2024-09-13T17:42:47.390368+0000","last_became_active":"2024-09-13T17:42:06.311843+0000","last_became_peered":"2024-09-13T17:42:06.311843+0000","last_unstale":"2024-09-13T17:42:47.390368+0000","last_undegraded":"2024-09-13T17:42:47.390368+0000","last_fullsized":"2024-09-13T17:42:47.390368+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:16.479088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.390569+0000","last_change":"2024-09-13T17:42:07.329208+0000","last_active":"2024-09-13T17:42:47.390569+0000","last_peered":"2024-09-13T17:42:47.390569+0000","last_clean":"2024-09-13T17:42:47.390569+0000","last_became_active":"2024-09-13T17:42:07.328947+0000","last_became_peered":"2024-09-13T17:42:07.328947+0000","last_unstale":"2024-09-13T17:42:47.390569+0000","last_undegraded":"2024-09-13T17:42:47.390569+0000","last_fullsized":"2024-09-13T17:42:47.390569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:25:39.337142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.389690+0000","last_change":"2024-09-13T17:42:02.137373+0000","last_active":"2024-09-13T17:42:47.389690+0000","last_peered":"2024-09-13T17:42:47.389690+0000","last_clean":"2024-09-13T17:42:47.389690+0000","last_became_active":"2024-09-13T17:42:00.116542+0000","last_became_peered":"2024-09-13T17:42:00.116542+0000","last_unstale":"2024-09-13T17:42:47.389690+0000","last_undegraded":"2024-09-13T17:42:47.389690+0000","last_fullsized":"2024-09-13T17:42:47.389690+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:53.754563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035063,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.395364+0000","last_change":"2024-09-13T17:42:06.304286+0000","last_active":"2024-09-13T17:42:47.395364+0000","last_peered":"2024-09-13T17:42:47.395364+0000","last_clean":"2024-09-13T17:42:47.395364+0000","last_became_active":"2024-09-13T17:42:06.304053+0000","last_became_peered":"2024-09-13T17:42:06.304053+0000","last_unstale":"2024-09-13T17:42:47.395364+0000","last_undegraded":"2024-09-13T17:42:47.395364+0000","last_fullsized":"2024-09-13T17:42:47.395364+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:20:52.415835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.396169+0000","last_change":"2024-09-13T17:42:07.326878+0000","last_active":"2024-09-13T17:42:47.396169+0000","last_peered":"2024-09-13T17:42:47.396169+0000","last_clean":"2024-09-13T17:42:47.396169+0000","last_became_active":"2024-09-13T17:42:07.326416+0000","last_became_peered":"2024-09-13T17:42:07.326416+0000","last_unstale":"2024-09-13T17:42:47.396169+0000","last_undegraded":"2024-09-13T17:42:47.396169+0000","last_fullsized":"2024-09-13T17:42:47.396169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:45.068616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.395066+0000","last_change":"2024-09-13T17:42:02.615831+0000","last_active":"2024-09-13T17:42:47.395066+0000","last_peered":"2024-09-13T17:42:47.395066+0000","last_clean":"2024-09-13T17:42:47.395066+0000","last_became_active":"2024-09-13T17:42:00.116427+0000","last_became_peered":"2024-09-13T17:42:00.116427+0000","last_unstale":"2024-09-13T17:42:47.395066+0000","last_undegraded":"2024-09-13T17:42:47.395066+0000","last_fullsized":"2024-09-13T17:42:47.395066+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:29:17.848348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000228423,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398773+0000","last_change":"2024-09-13T17:42:06.302978+0000","last_active":"2024-09-13T17:42:47.398773+0000","last_peered":"2024-09-13T17:42:47.398773+0000","last_clean":"2024-09-13T17:42:47.398773+0000","last_became_active":"2024-09-13T17:42:06.302485+0000","last_became_peered":"2024-09-13T17:42:06.302485+0000","last_unstale":"2024-09-13T17:42:47.398773+0000","last_undegraded":"2024-09-13T17:42:47.398773+0000","last_fullsized":"2024-09-13T17:42:47.398773+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:17:23.133953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398568+0000","last_change":"2024-09-13T17:42:07.318090+0000","last_active":"2024-09-13T17:42:47.398568+0000","last_peered":"2024-09-13T17:42:47.398568+0000","last_clean":"2024-09-13T17:42:47.398568+0000","last_became_active":"2024-09-13T17:42:07.317337+0000","last_became_peered":"2024-09-13T17:42:07.317337+0000","last_unstale":"2024-09-13T17:42:47.398568+0000","last_undegraded":"2024-09-13T17:42:47.398568+0000","last_fullsized":"2024-09-13T17:42:47.398568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:02:50.060149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.390458+0000","last_change":"2024-09-13T17:42:02.138081+0000","last_active":"2024-09-13T17:42:47.390458+0000","last_peered":"2024-09-13T17:42:47.390458+0000","last_clean":"2024-09-13T17:42:47.390458+0000","last_became_active":"2024-09-13T17:42:00.116465+0000","last_became_peered":"2024-09-13T17:42:00.116465+0000","last_unstale":"2024-09-13T17:42:47.390458+0000","last_undegraded":"2024-09-13T17:42:47.390458+0000","last_fullsized":"2024-09-13T17:42:47.390458+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:40:30.342531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001026573,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.396286+0000","last_change":"2024-09-13T17:42:06.304017+0000","last_active":"2024-09-13T17:42:47.396286+0000","last_peered":"2024-09-13T17:42:47.396286+0000","last_clean":"2024-09-13T17:42:47.396286+0000","last_became_active":"2024-09-13T17:42:06.303764+0000","last_became_peered":"2024-09-13T17:42:06.303764+0000","last_unstale":"2024-09-13T17:42:47.396286+0000","last_undegraded":"2024-09-13T17:42:47.396286+0000","last_fullsized":"2024-09-13T17:42:47.396286+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:20:51.407014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398260+0000","last_change":"2024-09-13T17:42:07.314438+0000","last_active":"2024-09-13T17:42:47.398260+0000","last_peered":"2024-09-13T17:42:47.398260+0000","last_clean":"2024-09-13T17:42:47.398260+0000","last_became_active":"2024-09-13T17:42:07.314196+0000","last_became_peered":"2024-09-13T17:42:07.314196+0000","last_unstale":"2024-09-13T17:42:47.398260+0000","last_undegraded":"2024-09-13T17:42:47.398260+0000","last_fullsized":"2024-09-13T17:42:47.398260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:33:57.543704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.395772+0000","last_change":"2024-09-13T17:42:06.304819+0000","last_active":"2024-09-13T17:42:47.395772+0000","last_peered":"2024-09-13T17:42:47.395772+0000","last_clean":"2024-09-13T17:42:47.395772+0000","last_became_active":"2024-09-13T17:42:06.304542+0000","last_became_peered":"2024-09-13T17:42:06.304542+0000","last_unstale":"2024-09-13T17:42:47.395772+0000","last_undegraded":"2024-09-13T17:42:47.395772+0000","last_fullsized":"2024-09-13T17:42:47.395772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:10:57.127180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398536+0000","last_change":"2024-09-13T17:42:02.135135+0000","last_active":"2024-09-13T17:42:47.398536+0000","last_peered":"2024-09-13T17:42:47.398536+0000","last_clean":"2024-09-13T17:42:47.398536+0000","last_became_active":"2024-09-13T17:42:00.364240+0000","last_became_peered":"2024-09-13T17:42:00.364240+0000","last_unstale":"2024-09-13T17:42:47.398536+0000","last_undegraded":"2024-09-13T17:42:47.398536+0000","last_fullsized":"2024-09-13T17:42:47.398536+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:31:29.382006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575944,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.397341+0000","last_change":"2024-09-13T17:42:07.338051+0000","last_active":"2024-09-13T17:42:47.397341+0000","last_peered":"2024-09-13T17:42:47.397341+0000","last_clean":"2024-09-13T17:42:47.397341+0000","last_became_active":"2024-09-13T17:42:07.337334+0000","last_became_peered":"2024-09-13T17:42:07.337334+0000","last_unstale":"2024-09-13T17:42:47.397341+0000","last_undegraded":"2024-09-13T17:42:47.397341+0000","last_fullsized":"2024-09-13T17:42:47.397341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:45.068616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398395+0000","last_change":"2024-09-13T17:42:06.300680+0000","last_active":"2024-09-13T17:42:47.398395+0000","last_peered":"2024-09-13T17:42:47.398395+0000","last_clean":"2024-09-13T17:42:47.398395+0000","last_became_active":"2024-09-13T17:42:06.299518+0000","last_became_peered":"2024-09-13T17:42:06.299518+0000","last_unstale":"2024-09-13T17:42:47.398395+0000","last_undegraded":"2024-09-13T17:42:47.398395+0000","last_fullsized":"2024-09-13T17:42:47.398395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:20:52.415835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.390916+0000","last_change":"2024-09-13T17:42:02.138136+0000","last_active":"2024-09-13T17:42:47.390916+0000","last_peered":"2024-09-13T17:42:47.390916+0000","last_clean":"2024-09-13T17:42:47.390916+0000","last_became_active":"2024-09-13T17:42:00.364233+0000","last_became_peered":"2024-09-13T17:42:00.364233+0000","last_unstale":"2024-09-13T17:42:47.390916+0000","last_undegraded":"2024-09-13T17:42:47.390916+0000","last_fullsized":"2024-09-13T17:42:47.390916+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:54.234194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010757130000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.395986+0000","last_change":"2024-09-13T17:42:07.315025+0000","last_active":"2024-09-13T17:42:47.395986+0000","last_peered":"2024-09-13T17:42:47.395986+0000","last_clean":"2024-09-13T17:42:47.395986+0000","last_became_active":"2024-09-13T17:42:07.314074+0000","last_became_peered":"2024-09-13T17:42:07.314074+0000","last_unstale":"2024-09-13T17:42:47.395986+0000","last_undegraded":"2024-09-13T17:42:47.395986+0000","last_fullsized":"2024-09-13T17:42:47.395986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:25.632002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.391805+0000","last_change":"2024-09-13T17:42:06.310475+0000","last_active":"2024-09-13T17:42:47.391805+0000","last_peered":"2024-09-13T17:42:47.391805+0000","last_clean":"2024-09-13T17:42:47.391805+0000","last_became_active":"2024-09-13T17:42:06.310230+0000","last_became_peered":"2024-09-13T17:42:06.310230+0000","last_unstale":"2024-09-13T17:42:47.391805+0000","last_undegraded":"2024-09-13T17:42:47.391805+0000","last_fullsized":"2024-09-13T17:42:47.391805+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:30:29.499136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.397944+0000","last_change":"2024-09-13T17:42:02.134963+0000","last_active":"2024-09-13T17:42:47.397944+0000","last_peered":"2024-09-13T17:42:47.397944+0000","last_clean":"2024-09-13T17:42:47.397944+0000","last_became_active":"2024-09-13T17:42:00.364233+0000","last_became_peered":"2024-09-13T17:42:00.364233+0000","last_unstale":"2024-09-13T17:42:47.397944+0000","last_undegraded":"2024-09-13T17:42:47.397944+0000","last_fullsized":"2024-09-13T17:42:47.397944+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:24:31.191067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393223,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:48.515573+0000","last_change":"2024-09-13T17:42:48.515573+0000","last_active":"2024-09-13T17:42:48.515573+0000","last_peered":"2024-09-13T17:42:48.515573+0000","last_clean":"2024-09-13T17:42:48.515573+0000","last_became_active":"2024-09-13T17:42:48.515160+0000","last_became_peered":"2024-09-13T17:42:48.515160+0000","last_unstale":"2024-09-13T17:42:48.515573+0000","last_undegraded":"2024-09-13T17:42:48.515573+0000","last_fullsized":"2024-09-13T17:42:48.515573+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:02.555469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"20'1","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.397988+0000","last_change":"2024-09-13T17:42:06.305416+0000","last_active":"2024-09-13T17:42:47.397988+0000","last_peered":"2024-09-13T17:42:47.397988+0000","last_clean":"2024-09-13T17:42:47.397988+0000","last_became_active":"2024-09-13T17:42:06.304782+0000","last_became_peered":"2024-09-13T17:42:06.304782+0000","last_unstale":"2024-09-13T17:42:47.397988+0000","last_undegraded":"2024-09-13T17:42:47.397988+0000","last_fullsized":"2024-09-13T17:42:47.397988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:08:01.276478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.391893+0000","last_change":"2024-09-13T17:42:02.138362+0000","last_active":"2024-09-13T17:42:47.391893+0000","last_peered":"2024-09-13T17:42:47.391893+0000","last_clean":"2024-09-13T17:42:47.391893+0000","last_became_active":"2024-09-13T17:42:00.364214+0000","last_became_peered":"2024-09-13T17:42:00.364214+0000","last_unstale":"2024-09-13T17:42:47.391893+0000","last_undegraded":"2024-09-13T17:42:47.391893+0000","last_fullsized":"2024-09-13T17:42:47.391893+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:33.976642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012776910000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.394953+0000","last_change":"2024-09-13T17:42:07.332265+0000","last_active":"2024-09-13T17:42:47.394953+0000","last_peered":"2024-09-13T17:42:47.394953+0000","last_clean":"2024-09-13T17:42:47.394953+0000","last_became_active":"2024-09-13T17:42:07.331834+0000","last_became_peered":"2024-09-13T17:42:07.331834+0000","last_unstale":"2024-09-13T17:42:47.394953+0000","last_undegraded":"2024-09-13T17:42:47.394953+0000","last_fullsized":"2024-09-13T17:42:47.394953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:52:45.662321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.394597+0000","last_change":"2024-09-13T17:42:07.333367+0000","last_active":"2024-09-13T17:42:47.394597+0000","last_peered":"2024-09-13T17:42:47.394597+0000","last_clean":"2024-09-13T17:42:47.394597+0000","last_became_active":"2024-09-13T17:42:07.333132+0000","last_became_peered":"2024-09-13T17:42:07.333132+0000","last_unstale":"2024-09-13T17:42:47.394597+0000","last_undegraded":"2024-09-13T17:42:47.394597+0000","last_fullsized":"2024-09-13T17:42:47.394597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:43.125769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"21'10","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.390581+0000","last_change":"2024-09-13T17:42:06.305919+0000","last_active":"2024-09-13T17:42:47.390581+0000","last_peered":"2024-09-13T17:42:47.390581+0000","last_clean":"2024-09-13T17:42:47.390581+0000","last_became_active":"2024-09-13T17:42:06.305663+0000","last_became_peered":"2024-09-13T17:42:06.305663+0000","last_unstale":"2024-09-13T17:42:47.390581+0000","last_undegraded":"2024-09-13T17:42:47.390581+0000","last_fullsized":"2024-09-13T17:42:47.390581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:24.628023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2941,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.397621+0000","last_change":"2024-09-13T17:42:07.330596+0000","last_active":"2024-09-13T17:42:47.397621+0000","last_peered":"2024-09-13T17:42:47.397621+0000","last_clean":"2024-09-13T17:42:47.397621+0000","last_became_active":"2024-09-13T17:42:07.330349+0000","last_became_peered":"2024-09-13T17:42:07.330349+0000","last_unstale":"2024-09-13T17:42:47.397621+0000","last_undegraded":"2024-09-13T17:42:47.397621+0000","last_fullsized":"2024-09-13T17:42:47.397621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:43.466051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.396518+0000","last_change":"2024-09-13T17:42:06.309276+0000","last_active":"2024-09-13T17:42:47.396518+0000","last_peered":"2024-09-13T17:42:47.396518+0000","last_clean":"2024-09-13T17:42:47.396518+0000","last_became_active":"2024-09-13T17:42:06.307311+0000","last_became_peered":"2024-09-13T17:42:06.307311+0000","last_unstale":"2024-09-13T17:42:47.396518+0000","last_undegraded":"2024-09-13T17:42:47.396518+0000","last_fullsized":"2024-09-13T17:42:47.396518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:31:06.878681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.395832+0000","last_change":"2024-09-13T17:42:06.301289+0000","last_active":"2024-09-13T17:42:47.395832+0000","last_peered":"2024-09-13T17:42:47.395832+0000","last_clean":"2024-09-13T17:42:47.395832+0000","last_became_active":"2024-09-13T17:42:06.301040+0000","last_became_peered":"2024-09-13T17:42:06.301040+0000","last_unstale":"2024-09-13T17:42:47.395832+0000","last_undegraded":"2024-09-13T17:42:47.395832+0000","last_fullsized":"2024-09-13T17:42:47.395832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:49:37.251198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.389714+0000","last_change":"2024-09-13T17:42:07.321515+0000","last_active":"2024-09-13T17:42:47.389714+0000","last_peered":"2024-09-13T17:42:47.389714+0000","last_clean":"2024-09-13T17:42:47.389714+0000","last_became_active":"2024-09-13T17:42:07.312719+0000","last_became_peered":"2024-09-13T17:42:07.312719+0000","last_unstale":"2024-09-13T17:42:47.389714+0000","last_undegraded":"2024-09-13T17:42:47.389714+0000","last_fullsized":"2024-09-13T17:42:47.389714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:29:48.289765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398273+0000","last_change":"2024-09-13T17:42:06.307317+0000","last_active":"2024-09-13T17:42:47.398273+0000","last_peered":"2024-09-13T17:42:47.398273+0000","last_clean":"2024-09-13T17:42:47.398273+0000","last_became_active":"2024-09-13T17:42:06.307087+0000","last_became_peered":"2024-09-13T17:42:06.307087+0000","last_unstale":"2024-09-13T17:42:47.398273+0000","last_undegraded":"2024-09-13T17:42:47.398273+0000","last_fullsized":"2024-09-13T17:42:47.398273+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:47:16.711770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.388844+0000","last_change":"2024-09-13T17:42:07.333996+0000","last_active":"2024-09-13T17:42:47.388844+0000","last_peered":"2024-09-13T17:42:47.388844+0000","last_clean":"2024-09-13T17:42:47.388844+0000","last_became_active":"2024-09-13T17:42:07.329707+0000","last_became_peered":"2024-09-13T17:42:07.329707+0000","last_unstale":"2024-09-13T17:42:47.388844+0000","last_undegraded":"2024-09-13T17:42:47.388844+0000","last_fullsized":"2024-09-13T17:42:47.388844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:01:57.881608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:48.403819+0000","last_change":"2024-09-13T17:42:48.403819+0000","last_active":"2024-09-13T17:42:48.403819+0000","last_peered":"2024-09-13T17:42:48.403819+0000","last_clean":"2024-09-13T17:42:48.403819+0000","last_became_active":"2024-09-13T17:42:48.403496+0000","last_became_peered":"2024-09-13T17:42:48.403496+0000","last_unstale":"2024-09-13T17:42:48.403819+0000","last_undegraded":"2024-09-13T17:42:48.403819+0000","last_fullsized":"2024-09-13T17:42:48.403819+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:14:07.564377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.389471+0000","last_change":"2024-09-13T17:42:06.308176+0000","last_active":"2024-09-13T17:42:47.389471+0000","last_peered":"2024-09-13T17:42:47.389471+0000","last_clean":"2024-09-13T17:42:47.389471+0000","last_became_active":"2024-09-13T17:42:06.307906+0000","last_became_peered":"2024-09-13T17:42:06.307906+0000","last_unstale":"2024-09-13T17:42:47.389471+0000","last_undegraded":"2024-09-13T17:42:47.389471+0000","last_fullsized":"2024-09-13T17:42:47.389471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:26:10.884606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.396250+0000","last_change":"2024-09-13T17:42:07.326636+0000","last_active":"2024-09-13T17:42:47.396250+0000","last_peered":"2024-09-13T17:42:47.396250+0000","last_clean":"2024-09-13T17:42:47.396250+0000","last_became_active":"2024-09-13T17:42:07.326146+0000","last_became_peered":"2024-09-13T17:42:07.326146+0000","last_unstale":"2024-09-13T17:42:47.396250+0000","last_undegraded":"2024-09-13T17:42:47.396250+0000","last_fullsized":"2024-09-13T17:42:47.396250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:22:40.880345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.397142+0000","last_change":"2024-09-13T17:42:06.312966+0000","last_active":"2024-09-13T17:42:47.397142+0000","last_peered":"2024-09-13T17:42:47.397142+0000","last_clean":"2024-09-13T17:42:47.397142+0000","last_became_active":"2024-09-13T17:42:06.312726+0000","last_became_peered":"2024-09-13T17:42:06.312726+0000","last_unstale":"2024-09-13T17:42:47.397142+0000","last_undegraded":"2024-09-13T17:42:47.397142+0000","last_fullsized":"2024-09-13T17:42:47.397142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:59.621392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.389919+0000","last_change":"2024-09-13T17:42:07.315788+0000","last_active":"2024-09-13T17:42:47.389919+0000","last_peered":"2024-09-13T17:42:47.389919+0000","last_clean":"2024-09-13T17:42:47.389919+0000","last_became_active":"2024-09-13T17:42:07.312751+0000","last_became_peered":"2024-09-13T17:42:07.312751+0000","last_unstale":"2024-09-13T17:42:47.389919+0000","last_undegraded":"2024-09-13T17:42:47.389919+0000","last_fullsized":"2024-09-13T17:42:47.389919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:24:19.756257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.397488+0000","last_change":"2024-09-13T17:42:06.303521+0000","last_active":"2024-09-13T17:42:47.397488+0000","last_peered":"2024-09-13T17:42:47.397488+0000","last_clean":"2024-09-13T17:42:47.397488+0000","last_became_active":"2024-09-13T17:42:06.303171+0000","last_became_peered":"2024-09-13T17:42:06.303171+0000","last_unstale":"2024-09-13T17:42:47.397488+0000","last_undegraded":"2024-09-13T17:42:47.397488+0000","last_fullsized":"2024-09-13T17:42:47.397488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:36:02.512243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.395792+0000","last_change":"2024-09-13T17:42:07.316113+0000","last_active":"2024-09-13T17:42:47.395792+0000","last_peered":"2024-09-13T17:42:47.395792+0000","last_clean":"2024-09-13T17:42:47.395792+0000","last_became_active":"2024-09-13T17:42:07.315546+0000","last_became_peered":"2024-09-13T17:42:07.315546+0000","last_unstale":"2024-09-13T17:42:47.395792+0000","last_undegraded":"2024-09-13T17:42:47.395792+0000","last_fullsized":"2024-09-13T17:42:47.395792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:30:30.503115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.396208+0000","last_change":"2024-09-13T17:42:06.305594+0000","last_active":"2024-09-13T17:42:47.396208+0000","last_peered":"2024-09-13T17:42:47.396208+0000","last_clean":"2024-09-13T17:42:47.396208+0000","last_became_active":"2024-09-13T17:42:06.305330+0000","last_became_peered":"2024-09-13T17:42:06.305330+0000","last_unstale":"2024-09-13T17:42:47.396208+0000","last_undegraded":"2024-09-13T17:42:47.396208+0000","last_fullsized":"2024-09-13T17:42:47.396208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:17:23.133953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.394552+0000","last_change":"2024-09-13T17:42:07.333147+0000","last_active":"2024-09-13T17:42:47.394552+0000","last_peered":"2024-09-13T17:42:47.394552+0000","last_clean":"2024-09-13T17:42:47.394552+0000","last_became_active":"2024-09-13T17:42:07.331637+0000","last_became_peered":"2024-09-13T17:42:07.331637+0000","last_unstale":"2024-09-13T17:42:47.394552+0000","last_undegraded":"2024-09-13T17:42:47.394552+0000","last_fullsized":"2024-09-13T17:42:47.394552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:25.632002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398331+0000","last_change":"2024-09-13T17:42:06.312084+0000","last_active":"2024-09-13T17:42:47.398331+0000","last_peered":"2024-09-13T17:42:47.398331+0000","last_clean":"2024-09-13T17:42:47.398331+0000","last_became_active":"2024-09-13T17:42:06.311848+0000","last_became_peered":"2024-09-13T17:42:06.311848+0000","last_unstale":"2024-09-13T17:42:47.398331+0000","last_undegraded":"2024-09-13T17:42:47.398331+0000","last_fullsized":"2024-09-13T17:42:47.398331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:32.379434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.395159+0000","last_change":"2024-09-13T17:42:07.333648+0000","last_active":"2024-09-13T17:42:47.395159+0000","last_peered":"2024-09-13T17:42:47.395159+0000","last_clean":"2024-09-13T17:42:47.395159+0000","last_became_active":"2024-09-13T17:42:07.332563+0000","last_became_peered":"2024-09-13T17:42:07.332563+0000","last_unstale":"2024-09-13T17:42:47.395159+0000","last_undegraded":"2024-09-13T17:42:47.395159+0000","last_fullsized":"2024-09-13T17:42:47.395159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:06.934321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.390460+0000","last_change":"2024-09-13T17:42:06.308815+0000","last_active":"2024-09-13T17:42:47.390460+0000","last_peered":"2024-09-13T17:42:47.390460+0000","last_clean":"2024-09-13T17:42:47.390460+0000","last_became_active":"2024-09-13T17:42:06.308407+0000","last_became_peered":"2024-09-13T17:42:06.308407+0000","last_unstale":"2024-09-13T17:42:47.390460+0000","last_undegraded":"2024-09-13T17:42:47.390460+0000","last_fullsized":"2024-09-13T17:42:47.390460+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:19.237563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.390044+0000","last_change":"2024-09-13T17:42:07.315582+0000","last_active":"2024-09-13T17:42:47.390044+0000","last_peered":"2024-09-13T17:42:47.390044+0000","last_clean":"2024-09-13T17:42:47.390044+0000","last_became_active":"2024-09-13T17:42:07.312277+0000","last_became_peered":"2024-09-13T17:42:07.312277+0000","last_unstale":"2024-09-13T17:42:47.390044+0000","last_undegraded":"2024-09-13T17:42:47.390044+0000","last_fullsized":"2024-09-13T17:42:47.390044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:59:50.392102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.390366+0000","last_change":"2024-09-13T17:42:06.312228+0000","last_active":"2024-09-13T17:42:47.390366+0000","last_peered":"2024-09-13T17:42:47.390366+0000","last_clean":"2024-09-13T17:42:47.390366+0000","last_became_active":"2024-09-13T17:42:06.311930+0000","last_became_peered":"2024-09-13T17:42:06.311930+0000","last_unstale":"2024-09-13T17:42:47.390366+0000","last_undegraded":"2024-09-13T17:42:47.390366+0000","last_fullsized":"2024-09-13T17:42:47.390366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:16.993105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.395001+0000","last_change":"2024-09-13T17:42:07.330018+0000","last_active":"2024-09-13T17:42:47.395001+0000","last_peered":"2024-09-13T17:42:47.395001+0000","last_clean":"2024-09-13T17:42:47.395001+0000","last_became_active":"2024-09-13T17:42:07.329763+0000","last_became_peered":"2024-09-13T17:42:07.329763+0000","last_unstale":"2024-09-13T17:42:47.395001+0000","last_undegraded":"2024-09-13T17:42:47.395001+0000","last_fullsized":"2024-09-13T17:42:47.395001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:38:02.719445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398201+0000","last_change":"2024-09-13T17:42:06.305609+0000","last_active":"2024-09-13T17:42:47.398201+0000","last_peered":"2024-09-13T17:42:47.398201+0000","last_clean":"2024-09-13T17:42:47.398201+0000","last_became_active":"2024-09-13T17:42:06.304878+0000","last_became_peered":"2024-09-13T17:42:06.304878+0000","last_unstale":"2024-09-13T17:42:47.398201+0000","last_undegraded":"2024-09-13T17:42:47.398201+0000","last_fullsized":"2024-09-13T17:42:47.398201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:32.379434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3727,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705686,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297580,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074260,"statfs":{"total":96636764160,"available":96332042240,"internally_reserved":0,"allocated":225280,"data_stored":71128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:42:55 2024","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.314}]},{"osd":1,"last update":"Fri Sep 13 17:42:55 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.36299999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:42:55 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.5}]}]},{"osd":2,"up_from":9,"seq":38654705686,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297564,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074276,"statfs":{"total":96636764160,"available":96332058624,"internally_reserved":0,"allocated":208896,"data_stored":70915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:42:56 2024","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.44,"5min":0.44,"15min":0.44},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.34300000000000003}]},{"osd":1,"last update":"Fri Sep 13 17:42:56 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.317}]},{"osd":3,"last update":"Fri Sep 13 17:42:56 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.38900000000000001}]}]},{"osd":1,"up_from":9,"seq":38654705689,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298144,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073696,"statfs":{"total":96636764160,"available":96331464704,"internally_reserved":0,"allocated":806912,"data_stored":659023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:42:55 2024","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"last":0.316},{"interface":"front","average":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"last":0.33700000000000002}]},{"osd":2,"last update":"Fri Sep 13 17:42:55 2024","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.317,"15min":0.317},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"last":0.27400000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:42:55 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.372},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.39100000000000001}]}]},{"osd":0,"up_from":9,"seq":38654705687,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298144,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073696,"statfs":{"total":96636764160,"available":96331464704,"internally_reserved":0,"allocated":806912,"data_stored":659058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:42:56 2024","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.309,"15min":0.309},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"last":0.221},{"interface":"front","average":{"1min":0.312,"5min":0.312,"15min":0.312},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"last":0.27200000000000002}]},{"osd":2,"last update":"Fri Sep 13 17:42:56 2024","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.246},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"last":0.309}]},{"osd":3,"last update":"Fri Sep 13 17:42:56 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.26200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:43:35.816 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:43:35.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.941+0000 7f8db719d640 1 Processor -- start 2024-09-13T17:43:35.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.941+0000 7f8db719d640 1 -- start start 2024-09-13T17:43:35.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.942+0000 7f8db719d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8db009abb0 0x7f8db012c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.942+0000 7f8db719d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8db012c6a0 0x7f8db012cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.942+0000 7f8db719d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8db0096d80 0x7f8db0097200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.942+0000 7f8db719d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8db009af80 con 0x7f8db009abb0 2024-09-13T17:43:35.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.942+0000 7f8db719d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db012cff0 con 0x7f8db0096d80 2024-09-13T17:43:35.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.942+0000 7f8db719d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8db006a530 con 0x7f8db012c6a0 2024-09-13T17:43:35.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.942+0000 7f8daffff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8db012c6a0 0x7f8db012cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.942+0000 7f8db573a640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8db0096d80 0x7f8db0097200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.942+0000 7f8daffff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8db012c6a0 0x7f8db012cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47364/0 (socket says 172.21.15.102:47364) 2024-09-13T17:43:35.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.942+0000 7f8daffff640 1 -- 172.21.15.102:0/3211156101 learned_addr learned my addr 172.21.15.102:0/3211156101 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:35.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.942+0000 7f8db573a640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8db0096d80 0x7f8db0097200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:39896/0 (socket says 172.21.15.102:39896) 2024-09-13T17:43:35.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.942+0000 7f8db4f39640 1 --2- 172.21.15.102:0/3211156101 >> v2:172.21.15.102:3300/0 conn(0x7f8db009abb0 0x7f8db012c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.942+0000 7f8daffff640 1 -- 172.21.15.102:0/3211156101 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8db0096d80 msgr2=0x7f8db0097200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:35.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.942+0000 7f8daffff640 1 --2- 172.21.15.102:0/3211156101 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8db0096d80 0x7f8db0097200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.942+0000 7f8daffff640 1 -- 172.21.15.102:0/3211156101 >> v2:172.21.15.102:3300/0 conn(0x7f8db009abb0 msgr2=0x7f8db012c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:35.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.942+0000 7f8daffff640 1 --2- 172.21.15.102:0/3211156101 >> v2:172.21.15.102:3300/0 conn(0x7f8db009abb0 0x7f8db012c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.942+0000 7f8daffff640 1 -- 172.21.15.102:0/3211156101 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db0098be0 con 0x7f8db012c6a0 2024-09-13T17:43:35.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.943+0000 7f8daffff640 1 --2- 172.21.15.102:0/3211156101 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8db012c6a0 0x7f8db012cab0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8da0009f50 tx=0x7f8da0009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b67fb39219f7613 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:35.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.943+0000 7f8daf7fe640 1 -- 172.21.15.102:0/3211156101 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da0031400 con 0x7f8db012c6a0 2024-09-13T17:43:35.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.943+0000 7f8daf7fe640 1 -- 172.21.15.102:0/3211156101 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da002d050 con 0x7f8db012c6a0 2024-09-13T17:43:35.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.943+0000 7f8daf7fe640 1 -- 172.21.15.102:0/3211156101 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da00317d0 con 0x7f8db012c6a0 2024-09-13T17:43:35.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.944+0000 7f8db719d640 1 -- 172.21.15.102:0/3211156101 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8db012c6a0 msgr2=0x7f8db012cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.944+0000 7f8db719d640 1 --2- 172.21.15.102:0/3211156101 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8db012c6a0 0x7f8db012cab0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8da0009f50 tx=0x7f8da0009840 comp rx=0 tx=0).stop 2024-09-13T17:43:35.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.944+0000 7f8db719d640 1 -- 172.21.15.102:0/3211156101 shutdown_connections 2024-09-13T17:43:35.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.944+0000 7f8db719d640 1 --2- 172.21.15.102:0/3211156101 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8db0096d80 0x7f8db0097200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.944+0000 7f8db719d640 1 --2- 172.21.15.102:0/3211156101 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8db012c6a0 0x7f8db012cab0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.944+0000 7f8db719d640 1 --2- 172.21.15.102:0/3211156101 >> v2:172.21.15.102:3300/0 conn(0x7f8db009abb0 0x7f8db012c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.944+0000 7f8db719d640 1 -- 172.21.15.102:0/3211156101 >> 172.21.15.102:0/3211156101 conn(0x7f8db009e820 msgr2=0x7f8db009ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:35.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.944+0000 7f8db719d640 1 -- 172.21.15.102:0/3211156101 shutdown_connections 2024-09-13T17:43:35.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.944+0000 7f8db719d640 1 -- 172.21.15.102:0/3211156101 wait complete. 2024-09-13T17:43:35.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.945+0000 7f8db719d640 1 Processor -- start 2024-09-13T17:43:35.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.945+0000 7f8db719d640 1 -- start start 2024-09-13T17:43:35.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.945+0000 7f8db719d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8db0096d80 0x7f8db0119410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.945+0000 7f8db719d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8db009abb0 0x7f8db0115c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.945+0000 7f8db719d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8db012c6a0 0x7f8db01161a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.945+0000 7f8db719d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db0117180 con 0x7f8db0096d80 2024-09-13T17:43:35.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.945+0000 7f8db719d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db009bd20 con 0x7f8db009abb0 2024-09-13T17:43:35.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.945+0000 7f8db719d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8db00981d0 con 0x7f8db012c6a0 2024-09-13T17:43:35.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.945+0000 7f8db4f39640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8db0096d80 0x7f8db0119410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.945+0000 7f8daffff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8db009abb0 0x7f8db0115c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.945+0000 7f8db573a640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8db012c6a0 0x7f8db01161a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.945+0000 7f8daffff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8db009abb0 0x7f8db0115c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:39900/0 (socket says 172.21.15.102:39900) 2024-09-13T17:43:35.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.945+0000 7f8daffff640 1 -- 172.21.15.102:0/1725376444 learned_addr learned my addr 172.21.15.102:0/1725376444 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:35.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.946+0000 7f8db573a640 1 -- 172.21.15.102:0/1725376444 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8db009abb0 msgr2=0x7f8db0115c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.946+0000 7f8db573a640 1 --2- 172.21.15.102:0/1725376444 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8db009abb0 0x7f8db0115c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.946+0000 7f8db573a640 1 -- 172.21.15.102:0/1725376444 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8db0096d80 msgr2=0x7f8db0119410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.946+0000 7f8db573a640 1 --2- 172.21.15.102:0/1725376444 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8db0096d80 0x7f8db0119410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.946+0000 7f8db573a640 1 -- 172.21.15.102:0/1725376444 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da0009ae0 con 0x7f8db012c6a0 2024-09-13T17:43:35.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.946+0000 7f8db4f39640 1 --2- 172.21.15.102:0/1725376444 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8db0096d80 0x7f8db0119410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:35.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.946+0000 7f8daffff640 1 --2- 172.21.15.102:0/1725376444 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8db009abb0 0x7f8db0115c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:35.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.946+0000 7f8db573a640 1 --2- 172.21.15.102:0/1725376444 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8db012c6a0 0x7f8db01161a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8d9800e900 tx=0x7f8d980090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:35.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.946+0000 7f8dacff9640 1 -- 172.21.15.102:0/1725376444 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9800c940 con 0x7f8db012c6a0 2024-09-13T17:43:35.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.946+0000 7f8dacff9640 1 -- 172.21.15.102:0/1725376444 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d9800caa0 con 0x7f8db012c6a0 2024-09-13T17:43:35.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.946+0000 7f8db719d640 1 -- 172.21.15.102:0/1725376444 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db01159a0 con 0x7f8db012c6a0 2024-09-13T17:43:35.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.946+0000 7f8dacff9640 1 -- 172.21.15.102:0/1725376444 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9800ce60 con 0x7f8db012c6a0 2024-09-13T17:43:35.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.946+0000 7f8db719d640 1 -- 172.21.15.102:0/1725376444 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db0116880 con 0x7f8db012c6a0 2024-09-13T17:43:35.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.947+0000 7f8dacff9640 1 -- 172.21.15.102:0/1725376444 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f8d98013680 con 0x7f8db012c6a0 2024-09-13T17:43:35.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.947+0000 7f8dacff9640 1 -- 172.21.15.102:0/1725376444 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f8d98042140 con 0x7f8db012c6a0 2024-09-13T17:43:35.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.947+0000 7f8db719d640 1 -- 172.21.15.102:0/1725376444 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db0098e00 con 0x7f8db012c6a0 2024-09-13T17:43:35.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:35.951+0000 7f8dacff9640 1 -- 172.21.15.102:0/1725376444 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f8d98020020 con 0x7f8db012c6a0 2024-09-13T17:43:36.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.101+0000 7f8db719d640 1 -- 172.21.15.102:0/1725376444 --> v1:172.21.15.102:6816/34937 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8db0064000 con 0x7f8d90031160 2024-09-13T17:43:36.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.107+0000 7f8dacff9640 1 -- 172.21.15.102:0/1725376444 <== mgr.4101 v1:172.21.15.102:6816/34937 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+276350 (unknown 2965378022 0 280590427) 0x7f8db0064000 con 0x7f8d90031160 2024-09-13T17:43:36.112 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:43:36.113 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-09-13T17:43:36.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.110+0000 7f8db719d640 1 -- 172.21.15.102:0/1725376444 >> v1:172.21.15.102:6816/34937 conn(0x7f8d90031160 legacy=0x7f8d90033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:36.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.110+0000 7f8db719d640 1 -- 172.21.15.102:0/1725376444 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8db012c6a0 msgr2=0x7f8db01161a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:36.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.110+0000 7f8db719d640 1 --2- 172.21.15.102:0/1725376444 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8db012c6a0 0x7f8db01161a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8d9800e900 tx=0x7f8d980090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.110+0000 7f8db719d640 1 -- 172.21.15.102:0/1725376444 shutdown_connections 2024-09-13T17:43:36.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.110+0000 7f8db719d640 1 --2- 172.21.15.102:0/1725376444 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8db012c6a0 0x7f8db01161a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.110+0000 7f8db719d640 1 --2- 172.21.15.102:0/1725376444 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8db009abb0 0x7f8db0115c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.110+0000 7f8db719d640 1 --2- 172.21.15.102:0/1725376444 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8db0096d80 0x7f8db0119410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.110+0000 7f8db719d640 1 -- 172.21.15.102:0/1725376444 >> 172.21.15.102:0/1725376444 conn(0x7f8db009e820 msgr2=0x7f8db0099aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:36.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.110+0000 7f8db719d640 1 -- 172.21.15.102:0/1725376444 shutdown_connections 2024-09-13T17:43:36.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.110+0000 7f8db719d640 1 -- 172.21.15.102:0/1725376444 wait complete. 2024-09-13T17:43:36.122 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2024-09-13T17:43:34.433558+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594114,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":130,"num_write_kb":1125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":98,"ondisk_log_size":98,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1191432,"kb_used_data":2000,"kb_used_omap":6,"kb_used_meta":1189369,"kb_avail":376295928,"statfs":{"total":386547056640,"available":385327030272,"internally_reserved":0,"allocated":2048000,"data_stored":1460124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6816,"internal_metadata":1217914208},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003250"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.395985+0000","last_change":"2024-09-13T17:42:07.332154+0000","last_active":"2024-09-13T17:42:47.395985+0000","last_peered":"2024-09-13T17:42:47.395985+0000","last_clean":"2024-09-13T17:42:47.395985+0000","last_became_active":"2024-09-13T17:42:07.331874+0000","last_became_peered":"2024-09-13T17:42:07.331874+0000","last_unstale":"2024-09-13T17:42:47.395985+0000","last_undegraded":"2024-09-13T17:42:47.395985+0000","last_fullsized":"2024-09-13T17:42:47.395985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:10:22.943375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.396057+0000","last_change":"2024-09-13T17:42:07.326086+0000","last_active":"2024-09-13T17:42:47.396057+0000","last_peered":"2024-09-13T17:42:47.396057+0000","last_clean":"2024-09-13T17:42:47.396057+0000","last_became_active":"2024-09-13T17:42:07.325830+0000","last_became_peered":"2024-09-13T17:42:07.325830+0000","last_unstale":"2024-09-13T17:42:47.396057+0000","last_undegraded":"2024-09-13T17:42:47.396057+0000","last_fullsized":"2024-09-13T17:42:47.396057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:54.719512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.396454+0000","last_change":"2024-09-13T17:42:07.326953+0000","last_active":"2024-09-13T17:42:47.396454+0000","last_peered":"2024-09-13T17:42:47.396454+0000","last_clean":"2024-09-13T17:42:47.396454+0000","last_became_active":"2024-09-13T17:42:07.326717+0000","last_became_peered":"2024-09-13T17:42:07.326717+0000","last_unstale":"2024-09-13T17:42:47.396454+0000","last_undegraded":"2024-09-13T17:42:47.396454+0000","last_fullsized":"2024-09-13T17:42:47.396454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:20.647404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.396025+0000","last_change":"2024-09-13T17:42:07.326356+0000","last_active":"2024-09-13T17:42:47.396025+0000","last_peered":"2024-09-13T17:42:47.396025+0000","last_clean":"2024-09-13T17:42:47.396025+0000","last_became_active":"2024-09-13T17:42:07.326127+0000","last_became_peered":"2024-09-13T17:42:07.326127+0000","last_unstale":"2024-09-13T17:42:47.396025+0000","last_undegraded":"2024-09-13T17:42:47.396025+0000","last_fullsized":"2024-09-13T17:42:47.396025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:44.405426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.396070+0000","last_change":"2024-09-13T17:42:07.330092+0000","last_active":"2024-09-13T17:42:47.396070+0000","last_peered":"2024-09-13T17:42:47.396070+0000","last_clean":"2024-09-13T17:42:47.396070+0000","last_became_active":"2024-09-13T17:42:07.327658+0000","last_became_peered":"2024-09-13T17:42:07.327658+0000","last_unstale":"2024-09-13T17:42:47.396070+0000","last_undegraded":"2024-09-13T17:42:47.396070+0000","last_fullsized":"2024-09-13T17:42:47.396070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:24:02.920430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:48.399934+0000","last_change":"2024-09-13T17:42:48.399934+0000","last_active":"2024-09-13T17:42:48.399934+0000","last_peered":"2024-09-13T17:42:48.399934+0000","last_clean":"2024-09-13T17:42:48.399934+0000","last_became_active":"2024-09-13T17:42:48.399527+0000","last_became_peered":"2024-09-13T17:42:48.399527+0000","last_unstale":"2024-09-13T17:42:48.399934+0000","last_undegraded":"2024-09-13T17:42:48.399934+0000","last_fullsized":"2024-09-13T17:42:48.399934+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:03:44.489038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.389584+0000","last_change":"2024-09-13T17:42:07.307630+0000","last_active":"2024-09-13T17:42:47.389584+0000","last_peered":"2024-09-13T17:42:47.389584+0000","last_clean":"2024-09-13T17:42:47.389584+0000","last_became_active":"2024-09-13T17:42:07.307382+0000","last_became_peered":"2024-09-13T17:42:07.307382+0000","last_unstale":"2024-09-13T17:42:47.389584+0000","last_undegraded":"2024-09-13T17:42:47.389584+0000","last_fullsized":"2024-09-13T17:42:47.389584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:26:32.846287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398633+0000","last_change":"2024-09-13T17:42:07.320736+0000","last_active":"2024-09-13T17:42:47.398633+0000","last_peered":"2024-09-13T17:42:47.398633+0000","last_clean":"2024-09-13T17:42:47.398633+0000","last_became_active":"2024-09-13T17:42:07.319801+0000","last_became_peered":"2024-09-13T17:42:07.319801+0000","last_unstale":"2024-09-13T17:42:47.398633+0000","last_undegraded":"2024-09-13T17:42:47.398633+0000","last_fullsized":"2024-09-13T17:42:47.398633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:28:32.030397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398290+0000","last_change":"2024-09-13T17:42:07.335208+0000","last_active":"2024-09-13T17:42:47.398290+0000","last_peered":"2024-09-13T17:42:47.398290+0000","last_clean":"2024-09-13T17:42:47.398290+0000","last_became_active":"2024-09-13T17:42:07.334956+0000","last_became_peered":"2024-09-13T17:42:07.334956+0000","last_unstale":"2024-09-13T17:42:47.398290+0000","last_undegraded":"2024-09-13T17:42:47.398290+0000","last_fullsized":"2024-09-13T17:42:47.398290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:59:53.544086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.397884+0000","last_change":"2024-09-13T17:42:07.317057+0000","last_active":"2024-09-13T17:42:47.397884+0000","last_peered":"2024-09-13T17:42:47.397884+0000","last_clean":"2024-09-13T17:42:47.397884+0000","last_became_active":"2024-09-13T17:42:07.316429+0000","last_became_peered":"2024-09-13T17:42:07.316429+0000","last_unstale":"2024-09-13T17:42:47.397884+0000","last_undegraded":"2024-09-13T17:42:47.397884+0000","last_fullsized":"2024-09-13T17:42:47.397884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:59:53.544086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.397688+0000","last_change":"2024-09-13T17:42:07.321106+0000","last_active":"2024-09-13T17:42:47.397688+0000","last_peered":"2024-09-13T17:42:47.397688+0000","last_clean":"2024-09-13T17:42:47.397688+0000","last_became_active":"2024-09-13T17:42:07.320827+0000","last_became_peered":"2024-09-13T17:42:07.320827+0000","last_unstale":"2024-09-13T17:42:47.397688+0000","last_undegraded":"2024-09-13T17:42:47.397688+0000","last_fullsized":"2024-09-13T17:42:47.397688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:34:34.794230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.395720+0000","last_change":"2024-09-13T17:42:07.331865+0000","last_active":"2024-09-13T17:42:47.395720+0000","last_peered":"2024-09-13T17:42:47.395720+0000","last_clean":"2024-09-13T17:42:47.395720+0000","last_became_active":"2024-09-13T17:42:07.331419+0000","last_became_peered":"2024-09-13T17:42:07.331419+0000","last_unstale":"2024-09-13T17:42:47.395720+0000","last_undegraded":"2024-09-13T17:42:47.395720+0000","last_fullsized":"2024-09-13T17:42:47.395720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:48.155823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:48.400031+0000","last_change":"2024-09-13T17:42:48.400031+0000","last_active":"2024-09-13T17:42:48.400031+0000","last_peered":"2024-09-13T17:42:48.400031+0000","last_clean":"2024-09-13T17:42:48.400031+0000","last_became_active":"2024-09-13T17:42:48.399662+0000","last_became_peered":"2024-09-13T17:42:48.399662+0000","last_unstale":"2024-09-13T17:42:48.400031+0000","last_undegraded":"2024-09-13T17:42:48.400031+0000","last_fullsized":"2024-09-13T17:42:48.400031+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:09.715214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.397259+0000","last_change":"2024-09-13T17:42:07.324771+0000","last_active":"2024-09-13T17:42:47.397259+0000","last_peered":"2024-09-13T17:42:47.397259+0000","last_clean":"2024-09-13T17:42:47.397259+0000","last_became_active":"2024-09-13T17:42:07.323820+0000","last_became_peered":"2024-09-13T17:42:07.323820+0000","last_unstale":"2024-09-13T17:42:47.397259+0000","last_undegraded":"2024-09-13T17:42:47.397259+0000","last_fullsized":"2024-09-13T17:42:47.397259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:24:02.920430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.395285+0000","last_change":"2024-09-13T17:42:07.325827+0000","last_active":"2024-09-13T17:42:47.395285+0000","last_peered":"2024-09-13T17:42:47.395285+0000","last_clean":"2024-09-13T17:42:47.395285+0000","last_became_active":"2024-09-13T17:42:07.325589+0000","last_became_peered":"2024-09-13T17:42:07.325589+0000","last_unstale":"2024-09-13T17:42:47.395285+0000","last_undegraded":"2024-09-13T17:42:47.395285+0000","last_fullsized":"2024-09-13T17:42:47.395285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:43.466051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.388642+0000","last_change":"2024-09-13T17:42:07.315478+0000","last_active":"2024-09-13T17:42:47.388642+0000","last_peered":"2024-09-13T17:42:47.388642+0000","last_clean":"2024-09-13T17:42:47.388642+0000","last_became_active":"2024-09-13T17:42:07.315218+0000","last_became_peered":"2024-09-13T17:42:07.315218+0000","last_unstale":"2024-09-13T17:42:47.388642+0000","last_undegraded":"2024-09-13T17:42:47.388642+0000","last_fullsized":"2024-09-13T17:42:47.388642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:06:44.563276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398180+0000","last_change":"2024-09-13T17:42:07.328192+0000","last_active":"2024-09-13T17:42:47.398180+0000","last_peered":"2024-09-13T17:42:47.398180+0000","last_clean":"2024-09-13T17:42:47.398180+0000","last_became_active":"2024-09-13T17:42:07.327951+0000","last_became_peered":"2024-09-13T17:42:07.327951+0000","last_unstale":"2024-09-13T17:42:47.398180+0000","last_undegraded":"2024-09-13T17:42:47.398180+0000","last_fullsized":"2024-09-13T17:42:47.398180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:28:32.030397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.396085+0000","last_change":"2024-09-13T17:42:07.314816+0000","last_active":"2024-09-13T17:42:47.396085+0000","last_peered":"2024-09-13T17:42:47.396085+0000","last_clean":"2024-09-13T17:42:47.396085+0000","last_became_active":"2024-09-13T17:42:07.314464+0000","last_became_peered":"2024-09-13T17:42:07.314464+0000","last_unstale":"2024-09-13T17:42:47.396085+0000","last_undegraded":"2024-09-13T17:42:47.396085+0000","last_fullsized":"2024-09-13T17:42:47.396085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:17.483066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.395672+0000","last_change":"2024-09-13T17:42:07.332269+0000","last_active":"2024-09-13T17:42:47.395672+0000","last_peered":"2024-09-13T17:42:47.395672+0000","last_clean":"2024-09-13T17:42:47.395672+0000","last_became_active":"2024-09-13T17:42:07.332040+0000","last_became_peered":"2024-09-13T17:42:07.332040+0000","last_unstale":"2024-09-13T17:42:47.395672+0000","last_undegraded":"2024-09-13T17:42:47.395672+0000","last_fullsized":"2024-09-13T17:42:47.395672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:48:49.368747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398189+0000","last_change":"2024-09-13T17:42:07.335425+0000","last_active":"2024-09-13T17:42:47.398189+0000","last_peered":"2024-09-13T17:42:47.398189+0000","last_clean":"2024-09-13T17:42:47.398189+0000","last_became_active":"2024-09-13T17:42:07.335173+0000","last_became_peered":"2024-09-13T17:42:07.335173+0000","last_unstale":"2024-09-13T17:42:47.398189+0000","last_undegraded":"2024-09-13T17:42:47.398189+0000","last_fullsized":"2024-09-13T17:42:47.398189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:00:10.355609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.397591+0000","last_change":"2024-09-13T17:42:07.331183+0000","last_active":"2024-09-13T17:42:47.397591+0000","last_peered":"2024-09-13T17:42:47.397591+0000","last_clean":"2024-09-13T17:42:47.397591+0000","last_became_active":"2024-09-13T17:42:07.330936+0000","last_became_peered":"2024-09-13T17:42:07.330936+0000","last_unstale":"2024-09-13T17:42:47.397591+0000","last_undegraded":"2024-09-13T17:42:47.397591+0000","last_fullsized":"2024-09-13T17:42:47.397591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:42:38.505262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.396192+0000","last_change":"2024-09-13T17:42:07.325877+0000","last_active":"2024-09-13T17:42:47.396192+0000","last_peered":"2024-09-13T17:42:47.396192+0000","last_clean":"2024-09-13T17:42:47.396192+0000","last_became_active":"2024-09-13T17:42:07.325657+0000","last_became_peered":"2024-09-13T17:42:07.325657+0000","last_unstale":"2024-09-13T17:42:47.396192+0000","last_undegraded":"2024-09-13T17:42:47.396192+0000","last_fullsized":"2024-09-13T17:42:47.396192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:48.663311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.389614+0000","last_change":"2024-09-13T17:42:07.322059+0000","last_active":"2024-09-13T17:42:47.389614+0000","last_peered":"2024-09-13T17:42:47.389614+0000","last_clean":"2024-09-13T17:42:47.389614+0000","last_became_active":"2024-09-13T17:42:07.312536+0000","last_became_peered":"2024-09-13T17:42:07.312536+0000","last_unstale":"2024-09-13T17:42:47.389614+0000","last_undegraded":"2024-09-13T17:42:47.389614+0000","last_fullsized":"2024-09-13T17:42:47.389614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:14:08.780968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398684+0000","last_change":"2024-09-13T17:42:07.336557+0000","last_active":"2024-09-13T17:42:47.398684+0000","last_peered":"2024-09-13T17:42:47.398684+0000","last_clean":"2024-09-13T17:42:47.398684+0000","last_became_active":"2024-09-13T17:42:07.335820+0000","last_became_peered":"2024-09-13T17:42:07.335820+0000","last_unstale":"2024-09-13T17:42:47.398684+0000","last_undegraded":"2024-09-13T17:42:47.398684+0000","last_fullsized":"2024-09-13T17:42:47.398684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:20.647404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398429+0000","last_change":"2024-09-13T17:42:07.337695+0000","last_active":"2024-09-13T17:42:47.398429+0000","last_peered":"2024-09-13T17:42:47.398429+0000","last_clean":"2024-09-13T17:42:47.398429+0000","last_became_active":"2024-09-13T17:42:07.337419+0000","last_became_peered":"2024-09-13T17:42:07.337419+0000","last_unstale":"2024-09-13T17:42:47.398429+0000","last_undegraded":"2024-09-13T17:42:47.398429+0000","last_fullsized":"2024-09-13T17:42:47.398429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:22:40.880345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.397901+0000","last_change":"2024-09-13T17:42:07.315008+0000","last_active":"2024-09-13T17:42:47.397901+0000","last_peered":"2024-09-13T17:42:47.397901+0000","last_clean":"2024-09-13T17:42:47.397901+0000","last_became_active":"2024-09-13T17:42:07.313477+0000","last_became_peered":"2024-09-13T17:42:07.313477+0000","last_unstale":"2024-09-13T17:42:47.397901+0000","last_undegraded":"2024-09-13T17:42:47.397901+0000","last_fullsized":"2024-09-13T17:42:47.397901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:25.632002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:48.397637+0000","last_change":"2024-09-13T17:42:48.397637+0000","last_active":"2024-09-13T17:42:48.397637+0000","last_peered":"2024-09-13T17:42:48.397637+0000","last_clean":"2024-09-13T17:42:48.397637+0000","last_became_active":"2024-09-13T17:42:48.397311+0000","last_became_peered":"2024-09-13T17:42:48.397311+0000","last_unstale":"2024-09-13T17:42:48.397637+0000","last_undegraded":"2024-09-13T17:42:48.397637+0000","last_fullsized":"2024-09-13T17:42:48.397637+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:12:22.089693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398470+0000","last_change":"2024-09-13T17:42:07.313528+0000","last_active":"2024-09-13T17:42:47.398470+0000","last_peered":"2024-09-13T17:42:47.398470+0000","last_clean":"2024-09-13T17:42:47.398470+0000","last_became_active":"2024-09-13T17:42:07.313092+0000","last_became_peered":"2024-09-13T17:42:07.313092+0000","last_unstale":"2024-09-13T17:42:47.398470+0000","last_undegraded":"2024-09-13T17:42:47.398470+0000","last_fullsized":"2024-09-13T17:42:47.398470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:48:49.368747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.390660+0000","last_change":"2024-09-13T17:42:07.314816+0000","last_active":"2024-09-13T17:42:47.390660+0000","last_peered":"2024-09-13T17:42:47.390660+0000","last_clean":"2024-09-13T17:42:47.390660+0000","last_became_active":"2024-09-13T17:42:07.314574+0000","last_became_peered":"2024-09-13T17:42:07.314574+0000","last_unstale":"2024-09-13T17:42:47.390660+0000","last_undegraded":"2024-09-13T17:42:47.390660+0000","last_fullsized":"2024-09-13T17:42:47.390660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:03:35.976183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.396358+0000","last_change":"2024-09-13T17:42:07.328522+0000","last_active":"2024-09-13T17:42:47.396358+0000","last_peered":"2024-09-13T17:42:47.396358+0000","last_clean":"2024-09-13T17:42:47.396358+0000","last_became_active":"2024-09-13T17:42:07.327893+0000","last_became_peered":"2024-09-13T17:42:07.327893+0000","last_unstale":"2024-09-13T17:42:47.396358+0000","last_undegraded":"2024-09-13T17:42:47.396358+0000","last_fullsized":"2024-09-13T17:42:47.396358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:02:50.060149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.389675+0000","last_change":"2024-09-13T17:42:07.321900+0000","last_active":"2024-09-13T17:42:47.389675+0000","last_peered":"2024-09-13T17:42:47.389675+0000","last_clean":"2024-09-13T17:42:47.389675+0000","last_became_active":"2024-09-13T17:42:07.319230+0000","last_became_peered":"2024-09-13T17:42:07.319230+0000","last_unstale":"2024-09-13T17:42:47.389675+0000","last_undegraded":"2024-09-13T17:42:47.389675+0000","last_fullsized":"2024-09-13T17:42:47.389675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:50:46.442172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398698+0000","last_change":"2024-09-13T17:42:07.315846+0000","last_active":"2024-09-13T17:42:47.398698+0000","last_peered":"2024-09-13T17:42:47.398698+0000","last_clean":"2024-09-13T17:42:47.398698+0000","last_became_active":"2024-09-13T17:42:07.315513+0000","last_became_peered":"2024-09-13T17:42:07.315513+0000","last_unstale":"2024-09-13T17:42:47.398698+0000","last_undegraded":"2024-09-13T17:42:47.398698+0000","last_fullsized":"2024-09-13T17:42:47.398698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:56:21.343829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:48.515524+0000","last_change":"2024-09-13T17:42:48.515524+0000","last_active":"2024-09-13T17:42:48.515524+0000","last_peered":"2024-09-13T17:42:48.515524+0000","last_clean":"2024-09-13T17:42:48.515524+0000","last_became_active":"2024-09-13T17:42:48.515203+0000","last_became_peered":"2024-09-13T17:42:48.515203+0000","last_unstale":"2024-09-13T17:42:48.515524+0000","last_undegraded":"2024-09-13T17:42:48.515524+0000","last_fullsized":"2024-09-13T17:42:48.515524+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:43:52.887484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.390769+0000","last_change":"2024-09-13T17:42:06.308504+0000","last_active":"2024-09-13T17:42:47.390769+0000","last_peered":"2024-09-13T17:42:47.390769+0000","last_clean":"2024-09-13T17:42:47.390769+0000","last_became_active":"2024-09-13T17:42:06.308199+0000","last_became_peered":"2024-09-13T17:42:06.308199+0000","last_unstale":"2024-09-13T17:42:47.390769+0000","last_undegraded":"2024-09-13T17:42:47.390769+0000","last_fullsized":"2024-09-13T17:42:47.390769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:05.930342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.390353+0000","last_change":"2024-09-13T17:42:07.322597+0000","last_active":"2024-09-13T17:42:47.390353+0000","last_peered":"2024-09-13T17:42:47.390353+0000","last_clean":"2024-09-13T17:42:47.390353+0000","last_became_active":"2024-09-13T17:42:07.321945+0000","last_became_peered":"2024-09-13T17:42:07.321945+0000","last_unstale":"2024-09-13T17:42:47.390353+0000","last_undegraded":"2024-09-13T17:42:47.390353+0000","last_fullsized":"2024-09-13T17:42:47.390353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:03:28.132876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.396318+0000","last_change":"2024-09-13T17:42:06.306367+0000","last_active":"2024-09-13T17:42:47.396318+0000","last_peered":"2024-09-13T17:42:47.396318+0000","last_clean":"2024-09-13T17:42:47.396318+0000","last_became_active":"2024-09-13T17:42:06.306146+0000","last_became_peered":"2024-09-13T17:42:06.306146+0000","last_unstale":"2024-09-13T17:42:47.396318+0000","last_undegraded":"2024-09-13T17:42:47.396318+0000","last_fullsized":"2024-09-13T17:42:47.396318+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:45:01.750995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.396643+0000","last_change":"2024-09-13T17:42:07.329528+0000","last_active":"2024-09-13T17:42:47.396643+0000","last_peered":"2024-09-13T17:42:47.396643+0000","last_clean":"2024-09-13T17:42:47.396643+0000","last_became_active":"2024-09-13T17:42:07.329296+0000","last_became_peered":"2024-09-13T17:42:07.329296+0000","last_unstale":"2024-09-13T17:42:47.396643+0000","last_undegraded":"2024-09-13T17:42:47.396643+0000","last_fullsized":"2024-09-13T17:42:47.396643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:33:57.543704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.389827+0000","last_change":"2024-09-13T17:42:06.312798+0000","last_active":"2024-09-13T17:42:47.389827+0000","last_peered":"2024-09-13T17:42:47.389827+0000","last_clean":"2024-09-13T17:42:47.389827+0000","last_became_active":"2024-09-13T17:42:06.312388+0000","last_became_peered":"2024-09-13T17:42:06.312388+0000","last_unstale":"2024-09-13T17:42:47.389827+0000","last_undegraded":"2024-09-13T17:42:47.389827+0000","last_fullsized":"2024-09-13T17:42:47.389827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:56:20.339850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:48.514881+0000","last_change":"2024-09-13T17:42:48.514881+0000","last_active":"2024-09-13T17:42:48.514881+0000","last_peered":"2024-09-13T17:42:48.514881+0000","last_clean":"2024-09-13T17:42:48.514881+0000","last_became_active":"2024-09-13T17:42:48.514539+0000","last_became_peered":"2024-09-13T17:42:48.514539+0000","last_unstale":"2024-09-13T17:42:48.514881+0000","last_undegraded":"2024-09-13T17:42:48.514881+0000","last_fullsized":"2024-09-13T17:42:48.514881+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:14:13.836498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398776+0000","last_change":"2024-09-13T17:42:06.305806+0000","last_active":"2024-09-13T17:42:47.398776+0000","last_peered":"2024-09-13T17:42:47.398776+0000","last_clean":"2024-09-13T17:42:47.398776+0000","last_became_active":"2024-09-13T17:42:06.305576+0000","last_became_peered":"2024-09-13T17:42:06.305576+0000","last_unstale":"2024-09-13T17:42:47.398776+0000","last_undegraded":"2024-09-13T17:42:47.398776+0000","last_fullsized":"2024-09-13T17:42:47.398776+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:47:16.711770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398428+0000","last_change":"2024-09-13T17:42:06.304106+0000","last_active":"2024-09-13T17:42:47.398428+0000","last_peered":"2024-09-13T17:42:47.398428+0000","last_clean":"2024-09-13T17:42:47.398428+0000","last_became_active":"2024-09-13T17:42:06.303845+0000","last_became_peered":"2024-09-13T17:42:06.303845+0000","last_unstale":"2024-09-13T17:42:47.398428+0000","last_undegraded":"2024-09-13T17:42:47.398428+0000","last_fullsized":"2024-09-13T17:42:47.398428+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:36:02.512243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.388874+0000","last_change":"2024-09-13T17:42:07.331147+0000","last_active":"2024-09-13T17:42:47.388874+0000","last_peered":"2024-09-13T17:42:47.388874+0000","last_clean":"2024-09-13T17:42:47.388874+0000","last_became_active":"2024-09-13T17:42:07.329386+0000","last_became_peered":"2024-09-13T17:42:07.329386+0000","last_unstale":"2024-09-13T17:42:47.388874+0000","last_undegraded":"2024-09-13T17:42:47.388874+0000","last_fullsized":"2024-09-13T17:42:47.388874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:59:21.210970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.395917+0000","last_change":"2024-09-13T17:42:06.304872+0000","last_active":"2024-09-13T17:42:47.395917+0000","last_peered":"2024-09-13T17:42:47.395917+0000","last_clean":"2024-09-13T17:42:47.395917+0000","last_became_active":"2024-09-13T17:42:06.304639+0000","last_became_peered":"2024-09-13T17:42:06.304639+0000","last_unstale":"2024-09-13T17:42:47.395917+0000","last_undegraded":"2024-09-13T17:42:47.395917+0000","last_fullsized":"2024-09-13T17:42:47.395917+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:40.147410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.397065+0000","last_change":"2024-09-13T17:42:07.331235+0000","last_active":"2024-09-13T17:42:47.397065+0000","last_peered":"2024-09-13T17:42:47.397065+0000","last_clean":"2024-09-13T17:42:47.397065+0000","last_became_active":"2024-09-13T17:42:07.330822+0000","last_became_peered":"2024-09-13T17:42:07.330822+0000","last_unstale":"2024-09-13T17:42:47.397065+0000","last_undegraded":"2024-09-13T17:42:47.397065+0000","last_fullsized":"2024-09-13T17:42:47.397065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:38:04.013092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.390635+0000","last_change":"2024-09-13T17:42:06.310319+0000","last_active":"2024-09-13T17:42:47.390635+0000","last_peered":"2024-09-13T17:42:47.390635+0000","last_clean":"2024-09-13T17:42:47.390635+0000","last_became_active":"2024-09-13T17:42:06.310085+0000","last_became_peered":"2024-09-13T17:42:06.310085+0000","last_unstale":"2024-09-13T17:42:47.390635+0000","last_undegraded":"2024-09-13T17:42:47.390635+0000","last_fullsized":"2024-09-13T17:42:47.390635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:53.715533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.395288+0000","last_change":"2024-09-13T17:42:07.333488+0000","last_active":"2024-09-13T17:42:47.395288+0000","last_peered":"2024-09-13T17:42:47.395288+0000","last_clean":"2024-09-13T17:42:47.395288+0000","last_became_active":"2024-09-13T17:42:07.332200+0000","last_became_peered":"2024-09-13T17:42:07.332200+0000","last_unstale":"2024-09-13T17:42:47.395288+0000","last_undegraded":"2024-09-13T17:42:47.395288+0000","last_fullsized":"2024-09-13T17:42:47.395288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:52.666716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.396110+0000","last_change":"2024-09-13T17:42:06.308854+0000","last_active":"2024-09-13T17:42:47.396110+0000","last_peered":"2024-09-13T17:42:47.396110+0000","last_clean":"2024-09-13T17:42:47.396110+0000","last_became_active":"2024-09-13T17:42:06.307173+0000","last_became_peered":"2024-09-13T17:42:06.307173+0000","last_unstale":"2024-09-13T17:42:47.396110+0000","last_undegraded":"2024-09-13T17:42:47.396110+0000","last_fullsized":"2024-09-13T17:42:47.396110+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:59.621392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.395858+0000","last_change":"2024-09-13T17:42:07.327431+0000","last_active":"2024-09-13T17:42:47.395858+0000","last_peered":"2024-09-13T17:42:47.395858+0000","last_clean":"2024-09-13T17:42:47.395858+0000","last_became_active":"2024-09-13T17:42:07.327202+0000","last_became_peered":"2024-09-13T17:42:07.327202+0000","last_unstale":"2024-09-13T17:42:47.395858+0000","last_undegraded":"2024-09-13T17:42:47.395858+0000","last_fullsized":"2024-09-13T17:42:47.395858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:42:38.505262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.390949+0000","last_change":"2024-09-13T17:42:06.308795+0000","last_active":"2024-09-13T17:42:47.390949+0000","last_peered":"2024-09-13T17:42:47.390949+0000","last_clean":"2024-09-13T17:42:47.390949+0000","last_became_active":"2024-09-13T17:42:06.308557+0000","last_became_peered":"2024-09-13T17:42:06.308557+0000","last_unstale":"2024-09-13T17:42:47.390949+0000","last_undegraded":"2024-09-13T17:42:47.390949+0000","last_fullsized":"2024-09-13T17:42:47.390949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:51.662738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398005+0000","last_change":"2024-09-13T17:42:07.327849+0000","last_active":"2024-09-13T17:42:47.398005+0000","last_peered":"2024-09-13T17:42:47.398005+0000","last_clean":"2024-09-13T17:42:47.398005+0000","last_became_active":"2024-09-13T17:42:07.327578+0000","last_became_peered":"2024-09-13T17:42:07.327578+0000","last_unstale":"2024-09-13T17:42:47.398005+0000","last_undegraded":"2024-09-13T17:42:47.398005+0000","last_fullsized":"2024-09-13T17:42:47.398005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:34:34.794230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.390194+0000","last_change":"2024-09-13T17:42:06.310708+0000","last_active":"2024-09-13T17:42:47.390194+0000","last_peered":"2024-09-13T17:42:47.390194+0000","last_clean":"2024-09-13T17:42:47.390194+0000","last_became_active":"2024-09-13T17:42:06.310476+0000","last_became_peered":"2024-09-13T17:42:06.310476+0000","last_unstale":"2024-09-13T17:42:47.390194+0000","last_undegraded":"2024-09-13T17:42:47.390194+0000","last_fullsized":"2024-09-13T17:42:47.390194+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:10:21.939396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.390317+0000","last_change":"2024-09-13T17:42:07.317665+0000","last_active":"2024-09-13T17:42:47.390317+0000","last_peered":"2024-09-13T17:42:47.390317+0000","last_clean":"2024-09-13T17:42:47.390317+0000","last_became_active":"2024-09-13T17:42:07.317401+0000","last_became_peered":"2024-09-13T17:42:07.317401+0000","last_unstale":"2024-09-13T17:42:47.390317+0000","last_undegraded":"2024-09-13T17:42:47.390317+0000","last_fullsized":"2024-09-13T17:42:47.390317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:12:55.460668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.390117+0000","last_change":"2024-09-13T17:42:06.313488+0000","last_active":"2024-09-13T17:42:47.390117+0000","last_peered":"2024-09-13T17:42:47.390117+0000","last_clean":"2024-09-13T17:42:47.390117+0000","last_became_active":"2024-09-13T17:42:06.313241+0000","last_became_peered":"2024-09-13T17:42:06.313241+0000","last_unstale":"2024-09-13T17:42:47.390117+0000","last_undegraded":"2024-09-13T17:42:47.390117+0000","last_fullsized":"2024-09-13T17:42:47.390117+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:48:48.364768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.395702+0000","last_change":"2024-09-13T17:42:07.333605+0000","last_active":"2024-09-13T17:42:47.395702+0000","last_peered":"2024-09-13T17:42:47.395702+0000","last_clean":"2024-09-13T17:42:47.395702+0000","last_became_active":"2024-09-13T17:42:07.332509+0000","last_became_peered":"2024-09-13T17:42:07.332509+0000","last_unstale":"2024-09-13T17:42:47.395702+0000","last_undegraded":"2024-09-13T17:42:47.395702+0000","last_fullsized":"2024-09-13T17:42:47.395702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:26:11.888585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.389821+0000","last_change":"2024-09-13T17:42:06.310629+0000","last_active":"2024-09-13T17:42:47.389821+0000","last_peered":"2024-09-13T17:42:47.389821+0000","last_clean":"2024-09-13T17:42:47.389821+0000","last_became_active":"2024-09-13T17:42:06.310103+0000","last_became_peered":"2024-09-13T17:42:06.310103+0000","last_unstale":"2024-09-13T17:42:47.389821+0000","last_undegraded":"2024-09-13T17:42:47.389821+0000","last_fullsized":"2024-09-13T17:42:47.389821+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:42.121791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398502+0000","last_change":"2024-09-13T17:42:07.335957+0000","last_active":"2024-09-13T17:42:47.398502+0000","last_peered":"2024-09-13T17:42:47.398502+0000","last_clean":"2024-09-13T17:42:47.398502+0000","last_became_active":"2024-09-13T17:42:07.331187+0000","last_became_peered":"2024-09-13T17:42:07.331187+0000","last_unstale":"2024-09-13T17:42:47.398502+0000","last_undegraded":"2024-09-13T17:42:47.398502+0000","last_fullsized":"2024-09-13T17:42:47.398502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:44.405426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":87,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398917+0000","last_change":"2024-09-13T17:41:55.654539+0000","last_active":"2024-09-13T17:42:47.398917+0000","last_peered":"2024-09-13T17:42:47.398917+0000","last_clean":"2024-09-13T17:42:47.398917+0000","last_became_active":"2024-09-13T17:41:55.653536+0000","last_became_peered":"2024-09-13T17:41:55.653536+0000","last_unstale":"2024-09-13T17:42:47.398917+0000","last_undegraded":"2024-09-13T17:42:47.398917+0000","last_fullsized":"2024-09-13T17:42:47.398917+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:54.532965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:54.532965+0000","last_clean_scrub_stamp":"2024-09-13T17:41:54.532965+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:56.662588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.391053+0000","last_change":"2024-09-13T17:42:02.137356+0000","last_active":"2024-09-13T17:42:47.391053+0000","last_peered":"2024-09-13T17:42:47.391053+0000","last_clean":"2024-09-13T17:42:47.391053+0000","last_became_active":"2024-09-13T17:42:00.116534+0000","last_became_peered":"2024-09-13T17:42:00.116534+0000","last_unstale":"2024-09-13T17:42:47.391053+0000","last_undegraded":"2024-09-13T17:42:47.391053+0000","last_fullsized":"2024-09-13T17:42:47.391053+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:06.665067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000321799,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.390368+0000","last_change":"2024-09-13T17:42:06.312144+0000","last_active":"2024-09-13T17:42:47.390368+0000","last_peered":"2024-09-13T17:42:47.390368+0000","last_clean":"2024-09-13T17:42:47.390368+0000","last_became_active":"2024-09-13T17:42:06.311843+0000","last_became_peered":"2024-09-13T17:42:06.311843+0000","last_unstale":"2024-09-13T17:42:47.390368+0000","last_undegraded":"2024-09-13T17:42:47.390368+0000","last_fullsized":"2024-09-13T17:42:47.390368+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:16.479088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.390569+0000","last_change":"2024-09-13T17:42:07.329208+0000","last_active":"2024-09-13T17:42:47.390569+0000","last_peered":"2024-09-13T17:42:47.390569+0000","last_clean":"2024-09-13T17:42:47.390569+0000","last_became_active":"2024-09-13T17:42:07.328947+0000","last_became_peered":"2024-09-13T17:42:07.328947+0000","last_unstale":"2024-09-13T17:42:47.390569+0000","last_undegraded":"2024-09-13T17:42:47.390569+0000","last_fullsized":"2024-09-13T17:42:47.390569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:25:39.337142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.389690+0000","last_change":"2024-09-13T17:42:02.137373+0000","last_active":"2024-09-13T17:42:47.389690+0000","last_peered":"2024-09-13T17:42:47.389690+0000","last_clean":"2024-09-13T17:42:47.389690+0000","last_became_active":"2024-09-13T17:42:00.116542+0000","last_became_peered":"2024-09-13T17:42:00.116542+0000","last_unstale":"2024-09-13T17:42:47.389690+0000","last_undegraded":"2024-09-13T17:42:47.389690+0000","last_fullsized":"2024-09-13T17:42:47.389690+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:53.754563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035063,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.395364+0000","last_change":"2024-09-13T17:42:06.304286+0000","last_active":"2024-09-13T17:42:47.395364+0000","last_peered":"2024-09-13T17:42:47.395364+0000","last_clean":"2024-09-13T17:42:47.395364+0000","last_became_active":"2024-09-13T17:42:06.304053+0000","last_became_peered":"2024-09-13T17:42:06.304053+0000","last_unstale":"2024-09-13T17:42:47.395364+0000","last_undegraded":"2024-09-13T17:42:47.395364+0000","last_fullsized":"2024-09-13T17:42:47.395364+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:20:52.415835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.396169+0000","last_change":"2024-09-13T17:42:07.326878+0000","last_active":"2024-09-13T17:42:47.396169+0000","last_peered":"2024-09-13T17:42:47.396169+0000","last_clean":"2024-09-13T17:42:47.396169+0000","last_became_active":"2024-09-13T17:42:07.326416+0000","last_became_peered":"2024-09-13T17:42:07.326416+0000","last_unstale":"2024-09-13T17:42:47.396169+0000","last_undegraded":"2024-09-13T17:42:47.396169+0000","last_fullsized":"2024-09-13T17:42:47.396169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:45.068616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.395066+0000","last_change":"2024-09-13T17:42:02.615831+0000","last_active":"2024-09-13T17:42:47.395066+0000","last_peered":"2024-09-13T17:42:47.395066+0000","last_clean":"2024-09-13T17:42:47.395066+0000","last_became_active":"2024-09-13T17:42:00.116427+0000","last_became_peered":"2024-09-13T17:42:00.116427+0000","last_unstale":"2024-09-13T17:42:47.395066+0000","last_undegraded":"2024-09-13T17:42:47.395066+0000","last_fullsized":"2024-09-13T17:42:47.395066+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:29:17.848348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000228423,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398773+0000","last_change":"2024-09-13T17:42:06.302978+0000","last_active":"2024-09-13T17:42:47.398773+0000","last_peered":"2024-09-13T17:42:47.398773+0000","last_clean":"2024-09-13T17:42:47.398773+0000","last_became_active":"2024-09-13T17:42:06.302485+0000","last_became_peered":"2024-09-13T17:42:06.302485+0000","last_unstale":"2024-09-13T17:42:47.398773+0000","last_undegraded":"2024-09-13T17:42:47.398773+0000","last_fullsized":"2024-09-13T17:42:47.398773+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:17:23.133953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398568+0000","last_change":"2024-09-13T17:42:07.318090+0000","last_active":"2024-09-13T17:42:47.398568+0000","last_peered":"2024-09-13T17:42:47.398568+0000","last_clean":"2024-09-13T17:42:47.398568+0000","last_became_active":"2024-09-13T17:42:07.317337+0000","last_became_peered":"2024-09-13T17:42:07.317337+0000","last_unstale":"2024-09-13T17:42:47.398568+0000","last_undegraded":"2024-09-13T17:42:47.398568+0000","last_fullsized":"2024-09-13T17:42:47.398568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:02:50.060149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.390458+0000","last_change":"2024-09-13T17:42:02.138081+0000","last_active":"2024-09-13T17:42:47.390458+0000","last_peered":"2024-09-13T17:42:47.390458+0000","last_clean":"2024-09-13T17:42:47.390458+0000","last_became_active":"2024-09-13T17:42:00.116465+0000","last_became_peered":"2024-09-13T17:42:00.116465+0000","last_unstale":"2024-09-13T17:42:47.390458+0000","last_undegraded":"2024-09-13T17:42:47.390458+0000","last_fullsized":"2024-09-13T17:42:47.390458+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:40:30.342531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001026573,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.396286+0000","last_change":"2024-09-13T17:42:06.304017+0000","last_active":"2024-09-13T17:42:47.396286+0000","last_peered":"2024-09-13T17:42:47.396286+0000","last_clean":"2024-09-13T17:42:47.396286+0000","last_became_active":"2024-09-13T17:42:06.303764+0000","last_became_peered":"2024-09-13T17:42:06.303764+0000","last_unstale":"2024-09-13T17:42:47.396286+0000","last_undegraded":"2024-09-13T17:42:47.396286+0000","last_fullsized":"2024-09-13T17:42:47.396286+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:20:51.407014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398260+0000","last_change":"2024-09-13T17:42:07.314438+0000","last_active":"2024-09-13T17:42:47.398260+0000","last_peered":"2024-09-13T17:42:47.398260+0000","last_clean":"2024-09-13T17:42:47.398260+0000","last_became_active":"2024-09-13T17:42:07.314196+0000","last_became_peered":"2024-09-13T17:42:07.314196+0000","last_unstale":"2024-09-13T17:42:47.398260+0000","last_undegraded":"2024-09-13T17:42:47.398260+0000","last_fullsized":"2024-09-13T17:42:47.398260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:33:57.543704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.395772+0000","last_change":"2024-09-13T17:42:06.304819+0000","last_active":"2024-09-13T17:42:47.395772+0000","last_peered":"2024-09-13T17:42:47.395772+0000","last_clean":"2024-09-13T17:42:47.395772+0000","last_became_active":"2024-09-13T17:42:06.304542+0000","last_became_peered":"2024-09-13T17:42:06.304542+0000","last_unstale":"2024-09-13T17:42:47.395772+0000","last_undegraded":"2024-09-13T17:42:47.395772+0000","last_fullsized":"2024-09-13T17:42:47.395772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:10:57.127180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398536+0000","last_change":"2024-09-13T17:42:02.135135+0000","last_active":"2024-09-13T17:42:47.398536+0000","last_peered":"2024-09-13T17:42:47.398536+0000","last_clean":"2024-09-13T17:42:47.398536+0000","last_became_active":"2024-09-13T17:42:00.364240+0000","last_became_peered":"2024-09-13T17:42:00.364240+0000","last_unstale":"2024-09-13T17:42:47.398536+0000","last_undegraded":"2024-09-13T17:42:47.398536+0000","last_fullsized":"2024-09-13T17:42:47.398536+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:31:29.382006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575944,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.397341+0000","last_change":"2024-09-13T17:42:07.338051+0000","last_active":"2024-09-13T17:42:47.397341+0000","last_peered":"2024-09-13T17:42:47.397341+0000","last_clean":"2024-09-13T17:42:47.397341+0000","last_became_active":"2024-09-13T17:42:07.337334+0000","last_became_peered":"2024-09-13T17:42:07.337334+0000","last_unstale":"2024-09-13T17:42:47.397341+0000","last_undegraded":"2024-09-13T17:42:47.397341+0000","last_fullsized":"2024-09-13T17:42:47.397341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:45.068616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398395+0000","last_change":"2024-09-13T17:42:06.300680+0000","last_active":"2024-09-13T17:42:47.398395+0000","last_peered":"2024-09-13T17:42:47.398395+0000","last_clean":"2024-09-13T17:42:47.398395+0000","last_became_active":"2024-09-13T17:42:06.299518+0000","last_became_peered":"2024-09-13T17:42:06.299518+0000","last_unstale":"2024-09-13T17:42:47.398395+0000","last_undegraded":"2024-09-13T17:42:47.398395+0000","last_fullsized":"2024-09-13T17:42:47.398395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:20:52.415835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.390916+0000","last_change":"2024-09-13T17:42:02.138136+0000","last_active":"2024-09-13T17:42:47.390916+0000","last_peered":"2024-09-13T17:42:47.390916+0000","last_clean":"2024-09-13T17:42:47.390916+0000","last_became_active":"2024-09-13T17:42:00.364233+0000","last_became_peered":"2024-09-13T17:42:00.364233+0000","last_unstale":"2024-09-13T17:42:47.390916+0000","last_undegraded":"2024-09-13T17:42:47.390916+0000","last_fullsized":"2024-09-13T17:42:47.390916+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:54.234194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010757130000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.395986+0000","last_change":"2024-09-13T17:42:07.315025+0000","last_active":"2024-09-13T17:42:47.395986+0000","last_peered":"2024-09-13T17:42:47.395986+0000","last_clean":"2024-09-13T17:42:47.395986+0000","last_became_active":"2024-09-13T17:42:07.314074+0000","last_became_peered":"2024-09-13T17:42:07.314074+0000","last_unstale":"2024-09-13T17:42:47.395986+0000","last_undegraded":"2024-09-13T17:42:47.395986+0000","last_fullsized":"2024-09-13T17:42:47.395986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:25.632002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.391805+0000","last_change":"2024-09-13T17:42:06.310475+0000","last_active":"2024-09-13T17:42:47.391805+0000","last_peered":"2024-09-13T17:42:47.391805+0000","last_clean":"2024-09-13T17:42:47.391805+0000","last_became_active":"2024-09-13T17:42:06.310230+0000","last_became_peered":"2024-09-13T17:42:06.310230+0000","last_unstale":"2024-09-13T17:42:47.391805+0000","last_undegraded":"2024-09-13T17:42:47.391805+0000","last_fullsized":"2024-09-13T17:42:47.391805+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:30:29.499136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.397944+0000","last_change":"2024-09-13T17:42:02.134963+0000","last_active":"2024-09-13T17:42:47.397944+0000","last_peered":"2024-09-13T17:42:47.397944+0000","last_clean":"2024-09-13T17:42:47.397944+0000","last_became_active":"2024-09-13T17:42:00.364233+0000","last_became_peered":"2024-09-13T17:42:00.364233+0000","last_unstale":"2024-09-13T17:42:47.397944+0000","last_undegraded":"2024-09-13T17:42:47.397944+0000","last_fullsized":"2024-09-13T17:42:47.397944+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:24:31.191067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393223,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:48.515573+0000","last_change":"2024-09-13T17:42:48.515573+0000","last_active":"2024-09-13T17:42:48.515573+0000","last_peered":"2024-09-13T17:42:48.515573+0000","last_clean":"2024-09-13T17:42:48.515573+0000","last_became_active":"2024-09-13T17:42:48.515160+0000","last_became_peered":"2024-09-13T17:42:48.515160+0000","last_unstale":"2024-09-13T17:42:48.515573+0000","last_undegraded":"2024-09-13T17:42:48.515573+0000","last_fullsized":"2024-09-13T17:42:48.515573+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:02.555469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"20'1","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.397988+0000","last_change":"2024-09-13T17:42:06.305416+0000","last_active":"2024-09-13T17:42:47.397988+0000","last_peered":"2024-09-13T17:42:47.397988+0000","last_clean":"2024-09-13T17:42:47.397988+0000","last_became_active":"2024-09-13T17:42:06.304782+0000","last_became_peered":"2024-09-13T17:42:06.304782+0000","last_unstale":"2024-09-13T17:42:47.397988+0000","last_undegraded":"2024-09-13T17:42:47.397988+0000","last_fullsized":"2024-09-13T17:42:47.397988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:08:01.276478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.391893+0000","last_change":"2024-09-13T17:42:02.138362+0000","last_active":"2024-09-13T17:42:47.391893+0000","last_peered":"2024-09-13T17:42:47.391893+0000","last_clean":"2024-09-13T17:42:47.391893+0000","last_became_active":"2024-09-13T17:42:00.364214+0000","last_became_peered":"2024-09-13T17:42:00.364214+0000","last_unstale":"2024-09-13T17:42:47.391893+0000","last_undegraded":"2024-09-13T17:42:47.391893+0000","last_fullsized":"2024-09-13T17:42:47.391893+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:33.976642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012776910000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.394953+0000","last_change":"2024-09-13T17:42:07.332265+0000","last_active":"2024-09-13T17:42:47.394953+0000","last_peered":"2024-09-13T17:42:47.394953+0000","last_clean":"2024-09-13T17:42:47.394953+0000","last_became_active":"2024-09-13T17:42:07.331834+0000","last_became_peered":"2024-09-13T17:42:07.331834+0000","last_unstale":"2024-09-13T17:42:47.394953+0000","last_undegraded":"2024-09-13T17:42:47.394953+0000","last_fullsized":"2024-09-13T17:42:47.394953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:52:45.662321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.394597+0000","last_change":"2024-09-13T17:42:07.333367+0000","last_active":"2024-09-13T17:42:47.394597+0000","last_peered":"2024-09-13T17:42:47.394597+0000","last_clean":"2024-09-13T17:42:47.394597+0000","last_became_active":"2024-09-13T17:42:07.333132+0000","last_became_peered":"2024-09-13T17:42:07.333132+0000","last_unstale":"2024-09-13T17:42:47.394597+0000","last_undegraded":"2024-09-13T17:42:47.394597+0000","last_fullsized":"2024-09-13T17:42:47.394597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:43.125769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"21'10","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.390581+0000","last_change":"2024-09-13T17:42:06.305919+0000","last_active":"2024-09-13T17:42:47.390581+0000","last_peered":"2024-09-13T17:42:47.390581+0000","last_clean":"2024-09-13T17:42:47.390581+0000","last_became_active":"2024-09-13T17:42:06.305663+0000","last_became_peered":"2024-09-13T17:42:06.305663+0000","last_unstale":"2024-09-13T17:42:47.390581+0000","last_undegraded":"2024-09-13T17:42:47.390581+0000","last_fullsized":"2024-09-13T17:42:47.390581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:24.628023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2941,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.397621+0000","last_change":"2024-09-13T17:42:07.330596+0000","last_active":"2024-09-13T17:42:47.397621+0000","last_peered":"2024-09-13T17:42:47.397621+0000","last_clean":"2024-09-13T17:42:47.397621+0000","last_became_active":"2024-09-13T17:42:07.330349+0000","last_became_peered":"2024-09-13T17:42:07.330349+0000","last_unstale":"2024-09-13T17:42:47.397621+0000","last_undegraded":"2024-09-13T17:42:47.397621+0000","last_fullsized":"2024-09-13T17:42:47.397621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:43.466051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.396518+0000","last_change":"2024-09-13T17:42:06.309276+0000","last_active":"2024-09-13T17:42:47.396518+0000","last_peered":"2024-09-13T17:42:47.396518+0000","last_clean":"2024-09-13T17:42:47.396518+0000","last_became_active":"2024-09-13T17:42:06.307311+0000","last_became_peered":"2024-09-13T17:42:06.307311+0000","last_unstale":"2024-09-13T17:42:47.396518+0000","last_undegraded":"2024-09-13T17:42:47.396518+0000","last_fullsized":"2024-09-13T17:42:47.396518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:31:06.878681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.395832+0000","last_change":"2024-09-13T17:42:06.301289+0000","last_active":"2024-09-13T17:42:47.395832+0000","last_peered":"2024-09-13T17:42:47.395832+0000","last_clean":"2024-09-13T17:42:47.395832+0000","last_became_active":"2024-09-13T17:42:06.301040+0000","last_became_peered":"2024-09-13T17:42:06.301040+0000","last_unstale":"2024-09-13T17:42:47.395832+0000","last_undegraded":"2024-09-13T17:42:47.395832+0000","last_fullsized":"2024-09-13T17:42:47.395832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:49:37.251198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.389714+0000","last_change":"2024-09-13T17:42:07.321515+0000","last_active":"2024-09-13T17:42:47.389714+0000","last_peered":"2024-09-13T17:42:47.389714+0000","last_clean":"2024-09-13T17:42:47.389714+0000","last_became_active":"2024-09-13T17:42:07.312719+0000","last_became_peered":"2024-09-13T17:42:07.312719+0000","last_unstale":"2024-09-13T17:42:47.389714+0000","last_undegraded":"2024-09-13T17:42:47.389714+0000","last_fullsized":"2024-09-13T17:42:47.389714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:29:48.289765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398273+0000","last_change":"2024-09-13T17:42:06.307317+0000","last_active":"2024-09-13T17:42:47.398273+0000","last_peered":"2024-09-13T17:42:47.398273+0000","last_clean":"2024-09-13T17:42:47.398273+0000","last_became_active":"2024-09-13T17:42:06.307087+0000","last_became_peered":"2024-09-13T17:42:06.307087+0000","last_unstale":"2024-09-13T17:42:47.398273+0000","last_undegraded":"2024-09-13T17:42:47.398273+0000","last_fullsized":"2024-09-13T17:42:47.398273+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:47:16.711770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.388844+0000","last_change":"2024-09-13T17:42:07.333996+0000","last_active":"2024-09-13T17:42:47.388844+0000","last_peered":"2024-09-13T17:42:47.388844+0000","last_clean":"2024-09-13T17:42:47.388844+0000","last_became_active":"2024-09-13T17:42:07.329707+0000","last_became_peered":"2024-09-13T17:42:07.329707+0000","last_unstale":"2024-09-13T17:42:47.388844+0000","last_undegraded":"2024-09-13T17:42:47.388844+0000","last_fullsized":"2024-09-13T17:42:47.388844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:01:57.881608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:48.403819+0000","last_change":"2024-09-13T17:42:48.403819+0000","last_active":"2024-09-13T17:42:48.403819+0000","last_peered":"2024-09-13T17:42:48.403819+0000","last_clean":"2024-09-13T17:42:48.403819+0000","last_became_active":"2024-09-13T17:42:48.403496+0000","last_became_peered":"2024-09-13T17:42:48.403496+0000","last_unstale":"2024-09-13T17:42:48.403819+0000","last_undegraded":"2024-09-13T17:42:48.403819+0000","last_fullsized":"2024-09-13T17:42:48.403819+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:14:07.564377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.389471+0000","last_change":"2024-09-13T17:42:06.308176+0000","last_active":"2024-09-13T17:42:47.389471+0000","last_peered":"2024-09-13T17:42:47.389471+0000","last_clean":"2024-09-13T17:42:47.389471+0000","last_became_active":"2024-09-13T17:42:06.307906+0000","last_became_peered":"2024-09-13T17:42:06.307906+0000","last_unstale":"2024-09-13T17:42:47.389471+0000","last_undegraded":"2024-09-13T17:42:47.389471+0000","last_fullsized":"2024-09-13T17:42:47.389471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:26:10.884606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.396250+0000","last_change":"2024-09-13T17:42:07.326636+0000","last_active":"2024-09-13T17:42:47.396250+0000","last_peered":"2024-09-13T17:42:47.396250+0000","last_clean":"2024-09-13T17:42:47.396250+0000","last_became_active":"2024-09-13T17:42:07.326146+0000","last_became_peered":"2024-09-13T17:42:07.326146+0000","last_unstale":"2024-09-13T17:42:47.396250+0000","last_undegraded":"2024-09-13T17:42:47.396250+0000","last_fullsized":"2024-09-13T17:42:47.396250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:22:40.880345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.397142+0000","last_change":"2024-09-13T17:42:06.312966+0000","last_active":"2024-09-13T17:42:47.397142+0000","last_peered":"2024-09-13T17:42:47.397142+0000","last_clean":"2024-09-13T17:42:47.397142+0000","last_became_active":"2024-09-13T17:42:06.312726+0000","last_became_peered":"2024-09-13T17:42:06.312726+0000","last_unstale":"2024-09-13T17:42:47.397142+0000","last_undegraded":"2024-09-13T17:42:47.397142+0000","last_fullsized":"2024-09-13T17:42:47.397142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:59.621392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.389919+0000","last_change":"2024-09-13T17:42:07.315788+0000","last_active":"2024-09-13T17:42:47.389919+0000","last_peered":"2024-09-13T17:42:47.389919+0000","last_clean":"2024-09-13T17:42:47.389919+0000","last_became_active":"2024-09-13T17:42:07.312751+0000","last_became_peered":"2024-09-13T17:42:07.312751+0000","last_unstale":"2024-09-13T17:42:47.389919+0000","last_undegraded":"2024-09-13T17:42:47.389919+0000","last_fullsized":"2024-09-13T17:42:47.389919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:24:19.756257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.397488+0000","last_change":"2024-09-13T17:42:06.303521+0000","last_active":"2024-09-13T17:42:47.397488+0000","last_peered":"2024-09-13T17:42:47.397488+0000","last_clean":"2024-09-13T17:42:47.397488+0000","last_became_active":"2024-09-13T17:42:06.303171+0000","last_became_peered":"2024-09-13T17:42:06.303171+0000","last_unstale":"2024-09-13T17:42:47.397488+0000","last_undegraded":"2024-09-13T17:42:47.397488+0000","last_fullsized":"2024-09-13T17:42:47.397488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:36:02.512243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.395792+0000","last_change":"2024-09-13T17:42:07.316113+0000","last_active":"2024-09-13T17:42:47.395792+0000","last_peered":"2024-09-13T17:42:47.395792+0000","last_clean":"2024-09-13T17:42:47.395792+0000","last_became_active":"2024-09-13T17:42:07.315546+0000","last_became_peered":"2024-09-13T17:42:07.315546+0000","last_unstale":"2024-09-13T17:42:47.395792+0000","last_undegraded":"2024-09-13T17:42:47.395792+0000","last_fullsized":"2024-09-13T17:42:47.395792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:30:30.503115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.396208+0000","last_change":"2024-09-13T17:42:06.305594+0000","last_active":"2024-09-13T17:42:47.396208+0000","last_peered":"2024-09-13T17:42:47.396208+0000","last_clean":"2024-09-13T17:42:47.396208+0000","last_became_active":"2024-09-13T17:42:06.305330+0000","last_became_peered":"2024-09-13T17:42:06.305330+0000","last_unstale":"2024-09-13T17:42:47.396208+0000","last_undegraded":"2024-09-13T17:42:47.396208+0000","last_fullsized":"2024-09-13T17:42:47.396208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:17:23.133953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.394552+0000","last_change":"2024-09-13T17:42:07.333147+0000","last_active":"2024-09-13T17:42:47.394552+0000","last_peered":"2024-09-13T17:42:47.394552+0000","last_clean":"2024-09-13T17:42:47.394552+0000","last_became_active":"2024-09-13T17:42:07.331637+0000","last_became_peered":"2024-09-13T17:42:07.331637+0000","last_unstale":"2024-09-13T17:42:47.394552+0000","last_undegraded":"2024-09-13T17:42:47.394552+0000","last_fullsized":"2024-09-13T17:42:47.394552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:25.632002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398331+0000","last_change":"2024-09-13T17:42:06.312084+0000","last_active":"2024-09-13T17:42:47.398331+0000","last_peered":"2024-09-13T17:42:47.398331+0000","last_clean":"2024-09-13T17:42:47.398331+0000","last_became_active":"2024-09-13T17:42:06.311848+0000","last_became_peered":"2024-09-13T17:42:06.311848+0000","last_unstale":"2024-09-13T17:42:47.398331+0000","last_undegraded":"2024-09-13T17:42:47.398331+0000","last_fullsized":"2024-09-13T17:42:47.398331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:32.379434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.395159+0000","last_change":"2024-09-13T17:42:07.333648+0000","last_active":"2024-09-13T17:42:47.395159+0000","last_peered":"2024-09-13T17:42:47.395159+0000","last_clean":"2024-09-13T17:42:47.395159+0000","last_became_active":"2024-09-13T17:42:07.332563+0000","last_became_peered":"2024-09-13T17:42:07.332563+0000","last_unstale":"2024-09-13T17:42:47.395159+0000","last_undegraded":"2024-09-13T17:42:47.395159+0000","last_fullsized":"2024-09-13T17:42:47.395159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:06.934321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.390460+0000","last_change":"2024-09-13T17:42:06.308815+0000","last_active":"2024-09-13T17:42:47.390460+0000","last_peered":"2024-09-13T17:42:47.390460+0000","last_clean":"2024-09-13T17:42:47.390460+0000","last_became_active":"2024-09-13T17:42:06.308407+0000","last_became_peered":"2024-09-13T17:42:06.308407+0000","last_unstale":"2024-09-13T17:42:47.390460+0000","last_undegraded":"2024-09-13T17:42:47.390460+0000","last_fullsized":"2024-09-13T17:42:47.390460+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:19.237563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.390044+0000","last_change":"2024-09-13T17:42:07.315582+0000","last_active":"2024-09-13T17:42:47.390044+0000","last_peered":"2024-09-13T17:42:47.390044+0000","last_clean":"2024-09-13T17:42:47.390044+0000","last_became_active":"2024-09-13T17:42:07.312277+0000","last_became_peered":"2024-09-13T17:42:07.312277+0000","last_unstale":"2024-09-13T17:42:47.390044+0000","last_undegraded":"2024-09-13T17:42:47.390044+0000","last_fullsized":"2024-09-13T17:42:47.390044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:59:50.392102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.390366+0000","last_change":"2024-09-13T17:42:06.312228+0000","last_active":"2024-09-13T17:42:47.390366+0000","last_peered":"2024-09-13T17:42:47.390366+0000","last_clean":"2024-09-13T17:42:47.390366+0000","last_became_active":"2024-09-13T17:42:06.311930+0000","last_became_peered":"2024-09-13T17:42:06.311930+0000","last_unstale":"2024-09-13T17:42:47.390366+0000","last_undegraded":"2024-09-13T17:42:47.390366+0000","last_fullsized":"2024-09-13T17:42:47.390366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:16.993105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.395001+0000","last_change":"2024-09-13T17:42:07.330018+0000","last_active":"2024-09-13T17:42:47.395001+0000","last_peered":"2024-09-13T17:42:47.395001+0000","last_clean":"2024-09-13T17:42:47.395001+0000","last_became_active":"2024-09-13T17:42:07.329763+0000","last_became_peered":"2024-09-13T17:42:07.329763+0000","last_unstale":"2024-09-13T17:42:47.395001+0000","last_undegraded":"2024-09-13T17:42:47.395001+0000","last_fullsized":"2024-09-13T17:42:47.395001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:06.280075+0000","last_clean_scrub_stamp":"2024-09-13T17:42:06.280075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:38:02.719445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:47.398201+0000","last_change":"2024-09-13T17:42:06.305609+0000","last_active":"2024-09-13T17:42:47.398201+0000","last_peered":"2024-09-13T17:42:47.398201+0000","last_clean":"2024-09-13T17:42:47.398201+0000","last_became_active":"2024-09-13T17:42:06.304878+0000","last_became_peered":"2024-09-13T17:42:06.304878+0000","last_unstale":"2024-09-13T17:42:47.398201+0000","last_undegraded":"2024-09-13T17:42:47.398201+0000","last_fullsized":"2024-09-13T17:42:47.398201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:05.276096+0000","last_clean_scrub_stamp":"2024-09-13T17:42:05.276096+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:32.379434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3727,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705686,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297580,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074260,"statfs":{"total":96636764160,"available":96332042240,"internally_reserved":0,"allocated":225280,"data_stored":71128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:42:55 2024","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.314}]},{"osd":1,"last update":"Fri Sep 13 17:42:55 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.36299999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:42:55 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.5}]}]},{"osd":2,"up_from":9,"seq":38654705686,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297564,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074276,"statfs":{"total":96636764160,"available":96332058624,"internally_reserved":0,"allocated":208896,"data_stored":70915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:42:56 2024","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.44,"5min":0.44,"15min":0.44},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.34300000000000003}]},{"osd":1,"last update":"Fri Sep 13 17:42:56 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.317}]},{"osd":3,"last update":"Fri Sep 13 17:42:56 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.38900000000000001}]}]},{"osd":1,"up_from":9,"seq":38654705689,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298144,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073696,"statfs":{"total":96636764160,"available":96331464704,"internally_reserved":0,"allocated":806912,"data_stored":659023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:42:55 2024","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"last":0.316},{"interface":"front","average":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"last":0.33700000000000002}]},{"osd":2,"last update":"Fri Sep 13 17:42:55 2024","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.317,"15min":0.317},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"last":0.27400000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:42:55 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.372},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.39100000000000001}]}]},{"osd":0,"up_from":9,"seq":38654705687,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298144,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073696,"statfs":{"total":96636764160,"available":96331464704,"internally_reserved":0,"allocated":806912,"data_stored":659058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:42:56 2024","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.309,"15min":0.309},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"last":0.221},{"interface":"front","average":{"1min":0.312,"5min":0.312,"15min":0.312},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"last":0.27200000000000002}]},{"osd":2,"last update":"Fri Sep 13 17:42:56 2024","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.246},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"last":0.309}]},{"osd":3,"last update":"Fri Sep 13 17:42:56 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.26200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:43:36.127 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T17:43:36.127 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-13T17:43:36.127 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-13T17:43:36.127 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-13T17:43:36.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.253+0000 7fddf8d9d640 1 Processor -- start 2024-09-13T17:43:36.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.253+0000 7fddf8d9d640 1 -- start start 2024-09-13T17:43:36.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.254+0000 7fddf8d9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fddf4068920 0x7fddf409d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:36.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.254+0000 7fddf8d9d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fddf409e020 0x7fddf412fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:36.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.254+0000 7fddf8d9d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddf40695a0 con 0x7fddf409dc50 2024-09-13T17:43:36.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.254+0000 7fddf8d9d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddf4068cf0 con 0x7fddf409e020 2024-09-13T17:43:36.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.254+0000 7fddf8d9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf409e4a0 con 0x7fddf4068920 2024-09-13T17:43:36.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.254+0000 7fddf2d9d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fddf409e020 0x7fddf412fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:36.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.254+0000 7fddf259c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fddf4068920 0x7fddf409d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:36.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.254+0000 7fddf1d9b640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fddf409dc50 0x7fddf412f310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:56278/0 (socket says 172.21.15.102:56278) 2024-09-13T17:43:36.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.254+0000 7fddf1d9b640 1 -- 172.21.15.102:0/1661805314 learned_addr learned my addr 172.21.15.102:0/1661805314 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:36.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.254+0000 7fddf259c640 1 -- 172.21.15.102:0/1661805314 >> v2:172.21.15.102:3300/0 conn(0x7fddf409e020 msgr2=0x7fddf412fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:36.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.254+0000 7fddf259c640 1 --2- 172.21.15.102:0/1661805314 >> v2:172.21.15.102:3300/0 conn(0x7fddf409e020 0x7fddf412fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.254+0000 7fddf259c640 1 -- 172.21.15.102:0/1661805314 >> v1:172.21.15.102:6789/0 conn(0x7fddf409dc50 legacy=0x7fddf412f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:36.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.254+0000 7fddf259c640 1 -- 172.21.15.102:0/1661805314 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf409f090 con 0x7fddf4068920 2024-09-13T17:43:36.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.254+0000 7fddf159a640 1 -- 172.21.15.102:0/1661805314 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 896827023 0 0) 0x7fddf40695a0 con 0x7fddf409dc50 2024-09-13T17:43:36.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.254+0000 7fddf2d9d640 1 --2- 172.21.15.102:0/1661805314 >> v2:172.21.15.102:3300/0 conn(0x7fddf409e020 0x7fddf412fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:36.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.255+0000 7fddf259c640 1 --2- 172.21.15.102:0/1661805314 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fddf4068920 0x7fddf409d710 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009850 tx=0x7fdde40090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aae857ef4c2b7495 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:36.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.255+0000 7fddf159a640 1 -- 172.21.15.102:0/1661805314 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde402ac80 con 0x7fddf4068920 2024-09-13T17:43:36.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.255+0000 7fddf159a640 1 -- 172.21.15.102:0/1661805314 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde402ade0 con 0x7fddf4068920 2024-09-13T17:43:36.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.255+0000 7fddf159a640 1 -- 172.21.15.102:0/1661805314 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde4032670 con 0x7fddf4068920 2024-09-13T17:43:36.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.256+0000 7fddf8d9d640 1 -- 172.21.15.102:0/1661805314 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fddf4068920 msgr2=0x7fddf409d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:36.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.256+0000 7fddf8d9d640 1 --2- 172.21.15.102:0/1661805314 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fddf4068920 0x7fddf409d710 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009850 tx=0x7fdde40090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.256+0000 7fddf8d9d640 1 -- 172.21.15.102:0/1661805314 shutdown_connections 2024-09-13T17:43:36.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.256+0000 7fddf8d9d640 1 --2- 172.21.15.102:0/1661805314 >> v2:172.21.15.102:3300/0 conn(0x7fddf409e020 0x7fddf412fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.256+0000 7fddf8d9d640 1 --2- 172.21.15.102:0/1661805314 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fddf4068920 0x7fddf409d710 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.256+0000 7fddf8d9d640 1 -- 172.21.15.102:0/1661805314 >> 172.21.15.102:0/1661805314 conn(0x7fddf406f320 msgr2=0x7fddf40a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:36.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.256+0000 7fddf8d9d640 1 -- 172.21.15.102:0/1661805314 shutdown_connections 2024-09-13T17:43:36.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.256+0000 7fddf8d9d640 1 -- 172.21.15.102:0/1661805314 wait complete. 2024-09-13T17:43:36.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.257+0000 7fddf8d9d640 1 Processor -- start 2024-09-13T17:43:36.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.257+0000 7fddf8d9d640 1 -- start start 2024-09-13T17:43:36.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.257+0000 7fddf8d9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fddf4068920 0x7fddf411af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:36.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.257+0000 7fddf8d9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fddf409dc50 0x7fddf411b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:36.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.257+0000 7fddf8d9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fddf409e020 0x7fddf413ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:36.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.257+0000 7fddf8d9d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf411acd0 con 0x7fddf409e020 2024-09-13T17:43:36.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.257+0000 7fddf8d9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf40a2110 con 0x7fddf4068920 2024-09-13T17:43:36.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.257+0000 7fddf8d9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fddf409e620 con 0x7fddf409dc50 2024-09-13T17:43:36.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.257+0000 7fddf259c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fddf4068920 0x7fddf411af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:36.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.257+0000 7fddf259c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fddf4068920 0x7fddf411af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:39918/0 (socket says 172.21.15.102:39918) 2024-09-13T17:43:36.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.257+0000 7fddf259c640 1 -- 172.21.15.102:0/2955600069 learned_addr learned my addr 172.21.15.102:0/2955600069 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:36.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.257+0000 7fddf1d9b640 1 --2- 172.21.15.102:0/2955600069 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fddf409dc50 0x7fddf411b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:36.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.257+0000 7fddf2d9d640 1 --2- 172.21.15.102:0/2955600069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fddf409e020 0x7fddf413ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:36.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.257+0000 7fddf259c640 1 -- 172.21.15.102:0/2955600069 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fddf409dc50 msgr2=0x7fddf411b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:36.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.257+0000 7fddf259c640 1 --2- 172.21.15.102:0/2955600069 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fddf409dc50 0x7fddf411b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.257+0000 7fddf259c640 1 -- 172.21.15.102:0/2955600069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fddf409e020 msgr2=0x7fddf413ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:36.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.257+0000 7fddf259c640 1 --2- 172.21.15.102:0/2955600069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fddf409e020 0x7fddf413ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.257+0000 7fddf259c640 1 -- 172.21.15.102:0/2955600069 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddec009a20 con 0x7fddf4068920 2024-09-13T17:43:36.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.257+0000 7fddf2d9d640 1 --2- 172.21.15.102:0/2955600069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fddf409e020 0x7fddf413ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:36.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.258+0000 7fddf259c640 1 --2- 172.21.15.102:0/2955600069 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fddf4068920 0x7fddf411af20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdde402a5b0 tx=0x7fdde4032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:36.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.258+0000 7fdde27fc640 1 -- 172.21.15.102:0/2955600069 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde403d720 con 0x7fddf4068920 2024-09-13T17:43:36.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.258+0000 7fdde27fc640 1 -- 172.21.15.102:0/2955600069 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde403d880 con 0x7fddf4068920 2024-09-13T17:43:36.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.258+0000 7fdde27fc640 1 -- 172.21.15.102:0/2955600069 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde403dc20 con 0x7fddf4068920 2024-09-13T17:43:36.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.258+0000 7fddf8d9d640 1 -- 172.21.15.102:0/2955600069 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdde4009390 con 0x7fddf4068920 2024-09-13T17:43:36.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.258+0000 7fddf8d9d640 1 -- 172.21.15.102:0/2955600069 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddf411e9d0 con 0x7fddf4068920 2024-09-13T17:43:36.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.258+0000 7fdde27fc640 1 -- 172.21.15.102:0/2955600069 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fdde4039040 con 0x7fddf4068920 2024-09-13T17:43:36.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.259+0000 7fdde27fc640 1 -- 172.21.15.102:0/2955600069 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7fdde40618b0 con 0x7fddf4068920 2024-09-13T17:43:36.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.259+0000 7fddf8d9d640 1 -- 172.21.15.102:0/2955600069 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddf411e430 con 0x7fddf4068920 2024-09-13T17:43:36.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.263+0000 7fdde27fc640 1 -- 172.21.15.102:0/2955600069 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fddf411e430 con 0x7fddf4068920 2024-09-13T17:43:36.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.446+0000 7fddf8d9d640 1 -- 172.21.15.102:0/2955600069 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fddf411e430 con 0x7fddf4068920 2024-09-13T17:43:36.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.446+0000 7fdde27fc640 1 -- 172.21.15.102:0/2955600069 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fddf411e430 con 0x7fddf4068920 2024-09-13T17:43:36.449 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:43:36.449 INFO:teuthology.orchestra.run.smithi102.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-13T17:43:36.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.448+0000 7fddf8d9d640 1 -- 172.21.15.102:0/2955600069 >> v1:172.21.15.102:6816/34937 conn(0x7fddd4031160 legacy=0x7fddd4033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:36.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.448+0000 7fddf8d9d640 1 -- 172.21.15.102:0/2955600069 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fddf4068920 msgr2=0x7fddf411af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:36.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.448+0000 7fddf8d9d640 1 --2- 172.21.15.102:0/2955600069 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fddf4068920 0x7fddf411af20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdde402a5b0 tx=0x7fdde4032000 comp rx=0 tx=0).stop 2024-09-13T17:43:36.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.448+0000 7fddf8d9d640 1 -- 172.21.15.102:0/2955600069 shutdown_connections 2024-09-13T17:43:36.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.448+0000 7fddf8d9d640 1 --2- 172.21.15.102:0/2955600069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fddf409e020 0x7fddf413ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.448+0000 7fddf8d9d640 1 --2- 172.21.15.102:0/2955600069 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fddf409dc50 0x7fddf411b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.448+0000 7fddf8d9d640 1 --2- 172.21.15.102:0/2955600069 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fddf4068920 0x7fddf411af20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.448+0000 7fddf8d9d640 1 -- 172.21.15.102:0/2955600069 >> 172.21.15.102:0/2955600069 conn(0x7fddf406f320 msgr2=0x7fddf41324e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:36.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.448+0000 7fddf8d9d640 1 -- 172.21.15.102:0/2955600069 shutdown_connections 2024-09-13T17:43:36.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.448+0000 7fddf8d9d640 1 -- 172.21.15.102:0/2955600069 wait complete. 2024-09-13T17:43:36.460 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-13T17:43:36.461 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:43:36.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.552+0000 7fd344f5d640 1 Processor -- start 2024-09-13T17:43:36.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.552+0000 7fd344f5d640 1 -- start start 2024-09-13T17:43:36.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.553+0000 7fd344f5d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd340068920 0x7fd34009d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:36.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.553+0000 7fd344f5d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd34009dc50 0x7fd34009e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:36.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.553+0000 7fd344f5d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd34012f310 0x7fd34012f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:36.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.553+0000 7fd344f5d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd340068cf0 con 0x7fd34012f310 2024-09-13T17:43:36.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.553+0000 7fd344f5d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3400695a0 con 0x7fd34009dc50 2024-09-13T17:43:36.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.553+0000 7fd344f5d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd34009f090 con 0x7fd340068920 2024-09-13T17:43:36.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.553+0000 7fd33ed9d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd34012f310 0x7fd34012f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:36.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.553+0000 7fd33dd9b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd34009dc50 0x7fd34009e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:36.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.553+0000 7fd33e59c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd340068920 0x7fd34009d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:36.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.553+0000 7fd33e59c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd340068920 0x7fd34009d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47378/0 (socket says 172.21.15.102:47378) 2024-09-13T17:43:36.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.553+0000 7fd33ed9d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd34012f310 0x7fd34012f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57642/0 (socket says 172.21.15.102:57642) 2024-09-13T17:43:36.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.553+0000 7fd33dd9b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd34009dc50 0x7fd34009e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:39922/0 (socket says 172.21.15.102:39922) 2024-09-13T17:43:36.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.553+0000 7fd33e59c640 1 -- 172.21.15.102:0/3580865064 learned_addr learned my addr 172.21.15.102:0/3580865064 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:36.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.553+0000 7fd33ed9d640 1 -- 172.21.15.102:0/3580865064 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd340068920 msgr2=0x7fd34009d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:36.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.553+0000 7fd33ed9d640 1 --2- 172.21.15.102:0/3580865064 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd340068920 0x7fd34009d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.553+0000 7fd33ed9d640 1 -- 172.21.15.102:0/3580865064 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd34009dc50 msgr2=0x7fd34009e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:36.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.553+0000 7fd33ed9d640 1 --2- 172.21.15.102:0/3580865064 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd34009dc50 0x7fd34009e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.553+0000 7fd33ed9d640 1 -- 172.21.15.102:0/3580865064 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd34009cce0 con 0x7fd34012f310 2024-09-13T17:43:36.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.553+0000 7fd33dd9b640 1 --2- 172.21.15.102:0/3580865064 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd34009dc50 0x7fd34009e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:36.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.553+0000 7fd33e59c640 1 --2- 172.21.15.102:0/3580865064 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd340068920 0x7fd34009d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:36.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.554+0000 7fd33ed9d640 1 --2- 172.21.15.102:0/3580865064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd34012f310 0x7fd34012f7b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd338009e60 tx=0x7fd338009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3930c52c7d00caa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:36.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.554+0000 7fd33d59a640 1 -- 172.21.15.102:0/3580865064 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd338007550 con 0x7fd34012f310 2024-09-13T17:43:36.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.554+0000 7fd33d59a640 1 -- 172.21.15.102:0/3580865064 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33802d050 con 0x7fd34012f310 2024-09-13T17:43:36.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.554+0000 7fd33d59a640 1 -- 172.21.15.102:0/3580865064 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd338007920 con 0x7fd34012f310 2024-09-13T17:43:36.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.555+0000 7fd344f5d640 1 -- 172.21.15.102:0/3580865064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd34012f310 msgr2=0x7fd34012f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:36.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.555+0000 7fd344f5d640 1 --2- 172.21.15.102:0/3580865064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd34012f310 0x7fd34012f7b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd338009e60 tx=0x7fd338009780 comp rx=0 tx=0).stop 2024-09-13T17:43:36.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.555+0000 7fd344f5d640 1 -- 172.21.15.102:0/3580865064 shutdown_connections 2024-09-13T17:43:36.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.555+0000 7fd344f5d640 1 --2- 172.21.15.102:0/3580865064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd34012f310 0x7fd34012f7b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.555+0000 7fd344f5d640 1 --2- 172.21.15.102:0/3580865064 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd34009dc50 0x7fd34009e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.555+0000 7fd344f5d640 1 --2- 172.21.15.102:0/3580865064 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd340068920 0x7fd34009d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.555+0000 7fd344f5d640 1 -- 172.21.15.102:0/3580865064 >> 172.21.15.102:0/3580865064 conn(0x7fd34006f320 msgr2=0x7fd3400a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:36.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.555+0000 7fd344f5d640 1 -- 172.21.15.102:0/3580865064 shutdown_connections 2024-09-13T17:43:36.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.555+0000 7fd344f5d640 1 -- 172.21.15.102:0/3580865064 wait complete. 2024-09-13T17:43:36.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.556+0000 7fd344f5d640 1 Processor -- start 2024-09-13T17:43:36.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.556+0000 7fd344f5d640 1 -- start start 2024-09-13T17:43:36.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.556+0000 7fd344f5d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd340068920 0x7fd34011f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:36.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.556+0000 7fd344f5d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd34009dc50 0x7fd34011bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:36.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.556+0000 7fd344f5d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd34012f310 0x7fd34011c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:36.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.556+0000 7fd344f5d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34011daf0 con 0x7fd34009dc50 2024-09-13T17:43:36.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.556+0000 7fd344f5d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd34011d380 con 0x7fd34012f310 2024-09-13T17:43:36.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.556+0000 7fd344f5d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd3400a2ae0 con 0x7fd340068920 2024-09-13T17:43:36.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.556+0000 7fd33e59c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd340068920 0x7fd34011f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:36.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.556+0000 7fd33dd9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd34009dc50 0x7fd34011bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:36.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.556+0000 7fd33ed9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd34012f310 0x7fd34011c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:36.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.556+0000 7fd33e59c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd340068920 0x7fd34011f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47380/0 (socket says 172.21.15.102:47380) 2024-09-13T17:43:36.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.556+0000 7fd33e59c640 1 -- 172.21.15.102:0/4052358219 learned_addr learned my addr 172.21.15.102:0/4052358219 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:36.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.556+0000 7fd33e59c640 1 -- 172.21.15.102:0/4052358219 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd34012f310 msgr2=0x7fd34011c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:36.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.557+0000 7fd33e59c640 1 --2- 172.21.15.102:0/4052358219 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd34012f310 0x7fd34011c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.557+0000 7fd33e59c640 1 -- 172.21.15.102:0/4052358219 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd34009dc50 msgr2=0x7fd34011bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:36.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.557+0000 7fd33e59c640 1 --2- 172.21.15.102:0/4052358219 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd34009dc50 0x7fd34011bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.557+0000 7fd33e59c640 1 -- 172.21.15.102:0/4052358219 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd328009450 con 0x7fd340068920 2024-09-13T17:43:36.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.557+0000 7fd33dd9b640 1 --2- 172.21.15.102:0/4052358219 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd34009dc50 0x7fd34011bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:36.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.557+0000 7fd33e59c640 1 --2- 172.21.15.102:0/4052358219 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd340068920 0x7fd34011f610 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd32800ec80 tx=0x7fd3280091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:36.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.557+0000 7fd3367fc640 1 -- 172.21.15.102:0/4052358219 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd328009730 con 0x7fd340068920 2024-09-13T17:43:36.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.557+0000 7fd3367fc640 1 -- 172.21.15.102:0/4052358219 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd32800eec0 con 0x7fd340068920 2024-09-13T17:43:36.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.557+0000 7fd344f5d640 1 -- 172.21.15.102:0/4052358219 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd330009390 con 0x7fd340068920 2024-09-13T17:43:36.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.557+0000 7fd3367fc640 1 -- 172.21.15.102:0/4052358219 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32800cdf0 con 0x7fd340068920 2024-09-13T17:43:36.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.557+0000 7fd344f5d640 1 -- 172.21.15.102:0/4052358219 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd338009a20 con 0x7fd340068920 2024-09-13T17:43:36.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.558+0000 7fd3367fc640 1 -- 172.21.15.102:0/4052358219 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fd328013680 con 0x7fd340068920 2024-09-13T17:43:36.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.558+0000 7fd3367fc640 1 -- 172.21.15.102:0/4052358219 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7fd3280424e0 con 0x7fd340068920 2024-09-13T17:43:36.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.558+0000 7fd344f5d640 1 -- 172.21.15.102:0/4052358219 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd340064000 con 0x7fd340068920 2024-09-13T17:43:36.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.562+0000 7fd3367fc640 1 -- 172.21.15.102:0/4052358219 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fd328021020 con 0x7fd340068920 2024-09-13T17:43:36.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.716+0000 7fd344f5d640 1 -- 172.21.15.102:0/4052358219 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3400a2f60 con 0x7fd340068920 2024-09-13T17:43:36.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.716+0000 7fd3367fc640 1 -- 172.21.15.102:0/4052358219 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+11660 (secure 0 0 0) 0x7fd3280236b0 con 0x7fd340068920 2024-09-13T17:43:36.718 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:43:36.718 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":23,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:42:48.385896+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:04.672265+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:05.583134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.17","mappings":[{"from":3,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]},{"pgid":"4.25","mappings":[{"from":3,"to":2}]},{"pgid":"4.33","mappings":[{"from":3,"to":0}]},{"pgid":"4.3a","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:43:36.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.718+0000 7fd344f5d640 1 -- 172.21.15.102:0/4052358219 >> v1:172.21.15.102:6816/34937 conn(0x7fd320031160 legacy=0x7fd320033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:36.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.718+0000 7fd344f5d640 1 -- 172.21.15.102:0/4052358219 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd340068920 msgr2=0x7fd34011f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:36.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.718+0000 7fd344f5d640 1 --2- 172.21.15.102:0/4052358219 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd340068920 0x7fd34011f610 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd32800ec80 tx=0x7fd3280091d0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.718+0000 7fd344f5d640 1 -- 172.21.15.102:0/4052358219 shutdown_connections 2024-09-13T17:43:36.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.718+0000 7fd344f5d640 1 --2- 172.21.15.102:0/4052358219 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd34012f310 0x7fd34011c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.718+0000 7fd344f5d640 1 --2- 172.21.15.102:0/4052358219 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd34009dc50 0x7fd34011bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.718+0000 7fd344f5d640 1 --2- 172.21.15.102:0/4052358219 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd340068920 0x7fd34011f610 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.718+0000 7fd344f5d640 1 -- 172.21.15.102:0/4052358219 >> 172.21.15.102:0/4052358219 conn(0x7fd34006f320 msgr2=0x7fd340098360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:36.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.718+0000 7fd344f5d640 1 -- 172.21.15.102:0/4052358219 shutdown_connections 2024-09-13T17:43:36.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.718+0000 7fd344f5d640 1 -- 172.21.15.102:0/4052358219 wait complete. 2024-09-13T17:43:36.731 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:54.442612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T17:41:59.076782+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 16, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:42:04.672265+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:42:05.583134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:43:36.731 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:43:36.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.825+0000 7f58324c9640 1 Processor -- start 2024-09-13T17:43:36.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.825+0000 7f58324c9640 1 -- start start 2024-09-13T17:43:36.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.825+0000 7f58324c9640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f582c097530 0x7f582c097900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:36.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.825+0000 7f58324c9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f582c098210 0x7f582c133640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:36.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.825+0000 7f58324c9640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f582c098c50 con 0x7f582c097e40 2024-09-13T17:43:36.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.825+0000 7f58324c9640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582c06ed00 con 0x7f582c097530 2024-09-13T17:43:36.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.825+0000 7f58324c9640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f582c068bd0 con 0x7f582c098210 2024-09-13T17:43:36.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.825+0000 7f582bfff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f582c097530 0x7f582c097900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:36.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.825+0000 7f582b7fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f582c097e40 0x7f582c12ff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:56290/0 (socket says 172.21.15.102:56290) 2024-09-13T17:43:36.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.825+0000 7f5830a66640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f582c098210 0x7f582c133640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:36.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.825+0000 7f582b7fe640 1 -- 172.21.15.102:0/2452603252 learned_addr learned my addr 172.21.15.102:0/2452603252 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:36.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.826+0000 7f582affd640 1 -- 172.21.15.102:0/2452603252 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1926114076 0 0) 0x7f582c098c50 con 0x7f582c097e40 2024-09-13T17:43:36.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.826+0000 7f5830a66640 1 -- 172.21.15.102:0/2452603252 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f582c097530 msgr2=0x7f582c097900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:36.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.826+0000 7f5830a66640 1 --2- 172.21.15.102:0/2452603252 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f582c097530 0x7f582c097900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.826+0000 7f5830a66640 1 -- 172.21.15.102:0/2452603252 >> v1:172.21.15.102:6789/0 conn(0x7f582c097e40 legacy=0x7f582c12ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:36.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.826+0000 7f5830a66640 1 -- 172.21.15.102:0/2452603252 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f582c099040 con 0x7f582c098210 2024-09-13T17:43:36.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.826+0000 7f582bfff640 1 --2- 172.21.15.102:0/2452603252 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f582c097530 0x7f582c097900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:36.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.826+0000 7f5830a66640 1 --2- 172.21.15.102:0/2452603252 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f582c098210 0x7f582c133640 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5824009e90 tx=0x7f5824009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=720379b43c4a48df server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:36.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.827+0000 7f582affd640 1 -- 172.21.15.102:0/2452603252 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5824031400 con 0x7f582c098210 2024-09-13T17:43:36.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.827+0000 7f582affd640 1 -- 172.21.15.102:0/2452603252 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582402d050 con 0x7f582c098210 2024-09-13T17:43:36.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.827+0000 7f582affd640 1 -- 172.21.15.102:0/2452603252 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5824031790 con 0x7f582c098210 2024-09-13T17:43:36.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.827+0000 7f58324c9640 1 -- 172.21.15.102:0/2452603252 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f582c098210 msgr2=0x7f582c133640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:36.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.827+0000 7f58324c9640 1 --2- 172.21.15.102:0/2452603252 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f582c098210 0x7f582c133640 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5824009e90 tx=0x7f5824009780 comp rx=0 tx=0).stop 2024-09-13T17:43:36.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.827+0000 7f58324c9640 1 -- 172.21.15.102:0/2452603252 shutdown_connections 2024-09-13T17:43:36.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.827+0000 7f58324c9640 1 --2- 172.21.15.102:0/2452603252 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f582c098210 0x7f582c133640 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.827+0000 7f58324c9640 1 --2- 172.21.15.102:0/2452603252 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f582c097530 0x7f582c097900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.827+0000 7f58324c9640 1 -- 172.21.15.102:0/2452603252 >> 172.21.15.102:0/2452603252 conn(0x7f582c09e720 msgr2=0x7f582c09eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:36.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.827+0000 7f58324c9640 1 -- 172.21.15.102:0/2452603252 shutdown_connections 2024-09-13T17:43:36.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.827+0000 7f58324c9640 1 -- 172.21.15.102:0/2452603252 wait complete. 2024-09-13T17:43:36.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.828+0000 7f58324c9640 1 Processor -- start 2024-09-13T17:43:36.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.828+0000 7f58324c9640 1 -- start start 2024-09-13T17:43:36.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.828+0000 7f58324c9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f582c097530 0x7f582c11e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:36.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.828+0000 7f58324c9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f582c097e40 0x7f582c11ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:36.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.828+0000 7f58324c9640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f582c098210 0x7f582c11b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:36.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.828+0000 7f58324c9640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c06e9e0 con 0x7f582c097e40 2024-09-13T17:43:36.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.828+0000 7f58324c9640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582c09deb0 con 0x7f582c098210 2024-09-13T17:43:36.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.828+0000 7f58324c9640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f582c11d910 con 0x7f582c097530 2024-09-13T17:43:36.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.829+0000 7f582bfff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f582c097530 0x7f582c11e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:36.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.829+0000 7f582bfff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f582c097530 0x7f582c11e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47394/0 (socket says 172.21.15.102:47394) 2024-09-13T17:43:36.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.829+0000 7f5830a66640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f582c098210 0x7f582c11b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:36.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.829+0000 7f582b7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f582c097e40 0x7f582c11ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:36.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.829+0000 7f582bfff640 1 -- 172.21.15.102:0/4054434214 learned_addr learned my addr 172.21.15.102:0/4054434214 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:36.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.829+0000 7f582bfff640 1 -- 172.21.15.102:0/4054434214 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f582c098210 msgr2=0x7f582c11b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:36.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.829+0000 7f582bfff640 1 --2- 172.21.15.102:0/4054434214 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f582c098210 0x7f582c11b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.829+0000 7f582bfff640 1 -- 172.21.15.102:0/4054434214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f582c097e40 msgr2=0x7f582c11ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:36.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.829+0000 7f582bfff640 1 --2- 172.21.15.102:0/4054434214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f582c097e40 0x7f582c11ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.829+0000 7f582bfff640 1 -- 172.21.15.102:0/4054434214 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581c00a530 con 0x7f582c097530 2024-09-13T17:43:36.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.829+0000 7f582bfff640 1 --2- 172.21.15.102:0/4054434214 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f582c097530 0x7f582c11e3b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f581c00ea40 tx=0x7f581c00a2b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:36.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.829+0000 7f5813fff640 1 -- 172.21.15.102:0/4054434214 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581c00cb10 con 0x7f582c097530 2024-09-13T17:43:36.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.829+0000 7f5813fff640 1 -- 172.21.15.102:0/4054434214 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f581c00cc70 con 0x7f582c097530 2024-09-13T17:43:36.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.829+0000 7f58324c9640 1 -- 172.21.15.102:0/4054434214 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5824009a20 con 0x7f582c097530 2024-09-13T17:43:36.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.829+0000 7f5813fff640 1 -- 172.21.15.102:0/4054434214 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581c01a430 con 0x7f582c097530 2024-09-13T17:43:36.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.829+0000 7f58324c9640 1 -- 172.21.15.102:0/4054434214 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582c11b6a0 con 0x7f582c097530 2024-09-13T17:43:36.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.830+0000 7f5813fff640 1 -- 172.21.15.102:0/4054434214 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f581c0113f0 con 0x7f582c097530 2024-09-13T17:43:36.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.830+0000 7f5813fff640 1 -- 172.21.15.102:0/4054434214 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f581c015030 con 0x7f582c097530 2024-09-13T17:43:36.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.830+0000 7f58324c9640 1 -- 172.21.15.102:0/4054434214 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f582c064000 con 0x7f582c097530 2024-09-13T17:43:36.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.834+0000 7f5813fff640 1 -- 172.21.15.102:0/4054434214 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f581c0193e0 con 0x7f582c097530 2024-09-13T17:43:36.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.986+0000 7f58324c9640 1 -- 172.21.15.102:0/4054434214 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f582c06dda0 con 0x7f582c097530 2024-09-13T17:43:36.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.986+0000 7f5813fff640 1 -- 172.21.15.102:0/4054434214 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f581c0212f0 con 0x7f582c097530 2024-09-13T17:43:36.988 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-09-13T17:43:36.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.988+0000 7f58324c9640 1 -- 172.21.15.102:0/4054434214 >> v1:172.21.15.102:6816/34937 conn(0x7f5818031160 legacy=0x7f5818033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:36.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.988+0000 7f58324c9640 1 -- 172.21.15.102:0/4054434214 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f582c097530 msgr2=0x7f582c11e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:36.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.988+0000 7f58324c9640 1 --2- 172.21.15.102:0/4054434214 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f582c097530 0x7f582c11e3b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f581c00ea40 tx=0x7f581c00a2b0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.988+0000 7f58324c9640 1 -- 172.21.15.102:0/4054434214 shutdown_connections 2024-09-13T17:43:36.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.988+0000 7f58324c9640 1 --2- 172.21.15.102:0/4054434214 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f582c098210 0x7f582c11b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.988+0000 7f58324c9640 1 --2- 172.21.15.102:0/4054434214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f582c097e40 0x7f582c11ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.988+0000 7f58324c9640 1 --2- 172.21.15.102:0/4054434214 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f582c097530 0x7f582c11e3b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.988+0000 7f58324c9640 1 -- 172.21.15.102:0/4054434214 >> 172.21.15.102:0/4054434214 conn(0x7f582c09e720 msgr2=0x7f582c09c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:36.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.988+0000 7f58324c9640 1 -- 172.21.15.102:0/4054434214 shutdown_connections 2024-09-13T17:43:36.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:36.988+0000 7f58324c9640 1 -- 172.21.15.102:0/4054434214 wait complete. 2024-09-13T17:43:37.000 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-13T17:43:37.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.093+0000 7f832ea57640 1 Processor -- start 2024-09-13T17:43:37.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.093+0000 7f832ea57640 1 -- start start 2024-09-13T17:43:37.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.094+0000 7f832ea57640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f83280689d0 0x7f832809d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:37.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.094+0000 7f832ea57640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f832809e190 0x7f832812fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:37.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.094+0000 7f832ea57640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8328096b10 con 0x7f832809dd70 2024-09-13T17:43:37.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.094+0000 7f832ea57640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8328068da0 con 0x7f83280689d0 2024-09-13T17:43:37.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.094+0000 7f832ea57640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8328096db0 con 0x7f832809e190 2024-09-13T17:43:37.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.094+0000 7f8327fff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f83280689d0 0x7f832809d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:37.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.094+0000 7f832cff4640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f832809e190 0x7f832812fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:37.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.094+0000 7f83277fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f832809dd70 0x7f832812f610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:56302/0 (socket says 172.21.15.102:56302) 2024-09-13T17:43:37.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.094+0000 7f8327fff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f83280689d0 0x7f832809d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57664/0 (socket says 172.21.15.102:57664) 2024-09-13T17:43:37.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.094+0000 7f83277fe640 1 -- 172.21.15.102:0/2240433469 learned_addr learned my addr 172.21.15.102:0/2240433469 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:37.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.094+0000 7f832cff4640 1 -- 172.21.15.102:0/2240433469 >> v2:172.21.15.102:3300/0 conn(0x7f83280689d0 msgr2=0x7f832809d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:37.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.094+0000 7f832cff4640 1 --2- 172.21.15.102:0/2240433469 >> v2:172.21.15.102:3300/0 conn(0x7f83280689d0 0x7f832809d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.094+0000 7f832cff4640 1 -- 172.21.15.102:0/2240433469 >> v1:172.21.15.102:6789/0 conn(0x7f832809dd70 legacy=0x7f832812f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:37.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.094+0000 7f832cff4640 1 -- 172.21.15.102:0/2240433469 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83280696e0 con 0x7f832809e190 2024-09-13T17:43:37.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.094+0000 7f8326ffd640 1 -- 172.21.15.102:0/2240433469 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1181107263 0 0) 0x7f8328096b10 con 0x7f832809dd70 2024-09-13T17:43:37.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.094+0000 7f8327fff640 1 --2- 172.21.15.102:0/2240433469 >> v2:172.21.15.102:3300/0 conn(0x7f83280689d0 0x7f832809d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:37.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.095+0000 7f832cff4640 1 --2- 172.21.15.102:0/2240433469 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f832809e190 0x7f832812fd20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8320009e60 tx=0x7f8320009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d45aec8be96d7c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:37.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.095+0000 7f8326ffd640 1 -- 172.21.15.102:0/2240433469 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8320007550 con 0x7f832809e190 2024-09-13T17:43:37.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.095+0000 7f8326ffd640 1 -- 172.21.15.102:0/2240433469 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832002d050 con 0x7f832809e190 2024-09-13T17:43:37.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.095+0000 7f8326ffd640 1 -- 172.21.15.102:0/2240433469 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83200078e0 con 0x7f832809e190 2024-09-13T17:43:37.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.096+0000 7f832ea57640 1 -- 172.21.15.102:0/2240433469 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f832809e190 msgr2=0x7f832812fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:37.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.096+0000 7f832ea57640 1 --2- 172.21.15.102:0/2240433469 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f832809e190 0x7f832812fd20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8320009e60 tx=0x7f8320009780 comp rx=0 tx=0).stop 2024-09-13T17:43:37.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.096+0000 7f832ea57640 1 -- 172.21.15.102:0/2240433469 shutdown_connections 2024-09-13T17:43:37.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.096+0000 7f832ea57640 1 --2- 172.21.15.102:0/2240433469 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f832809e190 0x7f832812fd20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.096+0000 7f832ea57640 1 --2- 172.21.15.102:0/2240433469 >> v2:172.21.15.102:3300/0 conn(0x7f83280689d0 0x7f832809d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.096+0000 7f832ea57640 1 -- 172.21.15.102:0/2240433469 >> 172.21.15.102:0/2240433469 conn(0x7f832806f320 msgr2=0x7f83280a2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:37.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.096+0000 7f832ea57640 1 -- 172.21.15.102:0/2240433469 shutdown_connections 2024-09-13T17:43:37.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.096+0000 7f832ea57640 1 -- 172.21.15.102:0/2240433469 wait complete. 2024-09-13T17:43:37.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.097+0000 7f832ea57640 1 Processor -- start 2024-09-13T17:43:37.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.097+0000 7f832ea57640 1 -- start start 2024-09-13T17:43:37.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.097+0000 7f832ea57640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f83280689d0 0x7f832811e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:37.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.097+0000 7f832ea57640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f832809dd70 0x7f832811ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:37.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.097+0000 7f832ea57640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f832809e190 0x7f832811f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:37.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.097+0000 7f832ea57640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8328121d80 con 0x7f832809dd70 2024-09-13T17:43:37.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.097+0000 7f832ea57640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832811fae0 con 0x7f83280689d0 2024-09-13T17:43:37.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.097+0000 7f832ea57640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8328120690 con 0x7f832809e190 2024-09-13T17:43:37.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.097+0000 7f8327fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f83280689d0 0x7f832811e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:37.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.097+0000 7f83277fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f832809dd70 0x7f832811ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:37.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.097+0000 7f8327fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f83280689d0 0x7f832811e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:39966/0 (socket says 172.21.15.102:39966) 2024-09-13T17:43:37.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.097+0000 7f8327fff640 1 -- 172.21.15.102:0/3530952250 learned_addr learned my addr 172.21.15.102:0/3530952250 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:37.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.097+0000 7f832cff4640 1 --2- 172.21.15.102:0/3530952250 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f832809e190 0x7f832811f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:37.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.098+0000 7f83277fe640 1 -- 172.21.15.102:0/3530952250 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f832809e190 msgr2=0x7f832811f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:37.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.098+0000 7f83277fe640 1 --2- 172.21.15.102:0/3530952250 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f832809e190 0x7f832811f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.098+0000 7f83277fe640 1 -- 172.21.15.102:0/3530952250 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f83280689d0 msgr2=0x7f832811e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:37.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.098+0000 7f83277fe640 1 --2- 172.21.15.102:0/3530952250 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f83280689d0 0x7f832811e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.098+0000 7f83277fe640 1 -- 172.21.15.102:0/3530952250 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8318009390 con 0x7f832809dd70 2024-09-13T17:43:37.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.098+0000 7f8327fff640 1 --2- 172.21.15.102:0/3530952250 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f83280689d0 0x7f832811e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:37.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.098+0000 7f83277fe640 1 --2- 172.21.15.102:0/3530952250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f832809dd70 0x7f832811ebb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f831c001eb0 tx=0x7f831c007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:37.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.098+0000 7f830ffff640 1 -- 172.21.15.102:0/3530952250 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c005aa0 con 0x7f832809dd70 2024-09-13T17:43:37.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.098+0000 7f830ffff640 1 -- 172.21.15.102:0/3530952250 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831c005c20 con 0x7f832809dd70 2024-09-13T17:43:37.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.098+0000 7f832ea57640 1 -- 172.21.15.102:0/3530952250 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8320009a20 con 0x7f832809dd70 2024-09-13T17:43:37.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.098+0000 7f830ffff640 1 -- 172.21.15.102:0/3530952250 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c0070d0 con 0x7f832809dd70 2024-09-13T17:43:37.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.098+0000 7f832ea57640 1 -- 172.21.15.102:0/3530952250 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8328120150 con 0x7f832809dd70 2024-09-13T17:43:37.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.099+0000 7f830ffff640 1 -- 172.21.15.102:0/3530952250 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f831c007500 con 0x7f832809dd70 2024-09-13T17:43:37.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.099+0000 7f830ffff640 1 -- 172.21.15.102:0/3530952250 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f831c041630 con 0x7f832809dd70 2024-09-13T17:43:37.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.099+0000 7f832ea57640 1 -- 172.21.15.102:0/3530952250 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8328064000 con 0x7f832809dd70 2024-09-13T17:43:37.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.103+0000 7f830ffff640 1 -- 172.21.15.102:0/3530952250 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f831c01a360 con 0x7f832809dd70 2024-09-13T17:43:37.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.252+0000 7f832ea57640 1 -- 172.21.15.102:0/3530952250 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f83280a39c0 con 0x7f832809dd70 2024-09-13T17:43:37.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.252+0000 7f830ffff640 1 -- 172.21.15.102:0/3530952250 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f831c017020 con 0x7f832809dd70 2024-09-13T17:43:37.254 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-09-13T17:43:37.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.254+0000 7f832ea57640 1 -- 172.21.15.102:0/3530952250 >> v1:172.21.15.102:6816/34937 conn(0x7f8314031160 legacy=0x7f8314033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:37.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.254+0000 7f832ea57640 1 -- 172.21.15.102:0/3530952250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f832809dd70 msgr2=0x7f832811ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:37.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.254+0000 7f832ea57640 1 --2- 172.21.15.102:0/3530952250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f832809dd70 0x7f832811ebb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f831c001eb0 tx=0x7f831c007c80 comp rx=0 tx=0).stop 2024-09-13T17:43:37.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.254+0000 7f832ea57640 1 -- 172.21.15.102:0/3530952250 shutdown_connections 2024-09-13T17:43:37.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.254+0000 7f832ea57640 1 --2- 172.21.15.102:0/3530952250 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f832809e190 0x7f832811f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.254+0000 7f832ea57640 1 --2- 172.21.15.102:0/3530952250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f832809dd70 0x7f832811ebb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.254+0000 7f832ea57640 1 --2- 172.21.15.102:0/3530952250 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f83280689d0 0x7f832811e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.254+0000 7f832ea57640 1 -- 172.21.15.102:0/3530952250 >> 172.21.15.102:0/3530952250 conn(0x7f832806f320 msgr2=0x7f83280a18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:37.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.254+0000 7f832ea57640 1 -- 172.21.15.102:0/3530952250 shutdown_connections 2024-09-13T17:43:37.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.254+0000 7f832ea57640 1 -- 172.21.15.102:0/3530952250 wait complete. 2024-09-13T17:43:37.266 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:43:37.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.359+0000 7fdd4c79d640 1 Processor -- start 2024-09-13T17:43:37.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.359+0000 7fdd4c79d640 1 -- start start 2024-09-13T17:43:37.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.360+0000 7fdd4c79d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdd4412f8d0 0x7fdd4412fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:37.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.360+0000 7fdd4c79d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fdd44068920 0x7fdd44068cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:37.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.360+0000 7fdd4c79d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdd44069230 0x7fdd44136720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:37.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.360+0000 7fdd4c79d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd44096d80 con 0x7fdd44068920 2024-09-13T17:43:37.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.360+0000 7fdd4c79d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd440a2840 con 0x7fdd44069230 2024-09-13T17:43:37.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.360+0000 7fdd4c79d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdd440a0e10 con 0x7fdd4412f8d0 2024-09-13T17:43:37.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.360+0000 7fdd4a539640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdd4412f8d0 0x7fdd4412fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:37.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.360+0000 7fdd49d38640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fdd44068920 0x7fdd44068cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:37.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.360+0000 7fdd4ad3a640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdd44069230 0x7fdd44136720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:37.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.360+0000 7fdd49d38640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fdd44068920 0x7fdd44068cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57690/0 (socket says 172.21.15.102:57690) 2024-09-13T17:43:37.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.360+0000 7fdd4ad3a640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdd44069230 0x7fdd44136720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:39974/0 (socket says 172.21.15.102:39974) 2024-09-13T17:43:37.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.360+0000 7fdd49d38640 1 -- 172.21.15.102:0/2700854962 learned_addr learned my addr 172.21.15.102:0/2700854962 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:37.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.360+0000 7fdd4a539640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdd4412f8d0 0x7fdd4412fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47416/0 (socket says 172.21.15.102:47416) 2024-09-13T17:43:37.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.360+0000 7fdd49d38640 1 -- 172.21.15.102:0/2700854962 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdd4412f8d0 msgr2=0x7fdd4412fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:37.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.360+0000 7fdd49d38640 1 --2- 172.21.15.102:0/2700854962 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdd4412f8d0 0x7fdd4412fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.360+0000 7fdd49d38640 1 -- 172.21.15.102:0/2700854962 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdd44069230 msgr2=0x7fdd44136720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:37.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.360+0000 7fdd49d38640 1 --2- 172.21.15.102:0/2700854962 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdd44069230 0x7fdd44136720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.360+0000 7fdd49d38640 1 -- 172.21.15.102:0/2700854962 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4412f6b0 con 0x7fdd44068920 2024-09-13T17:43:37.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.360+0000 7fdd4ad3a640 1 --2- 172.21.15.102:0/2700854962 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdd44069230 0x7fdd44136720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:37.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.360+0000 7fdd4a539640 1 --2- 172.21.15.102:0/2700854962 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdd4412f8d0 0x7fdd4412fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:37.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.361+0000 7fdd49d38640 1 --2- 172.21.15.102:0/2700854962 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd44068920 0x7fdd44068cf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdd34009910 tx=0x7fdd340091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba6917eba9a17a9f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:37.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.361+0000 7fdd49537640 1 -- 172.21.15.102:0/2700854962 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3402ac90 con 0x7fdd44068920 2024-09-13T17:43:37.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.361+0000 7fdd49537640 1 -- 172.21.15.102:0/2700854962 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3402adf0 con 0x7fdd44068920 2024-09-13T17:43:37.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.361+0000 7fdd49537640 1 -- 172.21.15.102:0/2700854962 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd34032670 con 0x7fdd44068920 2024-09-13T17:43:37.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.362+0000 7fdd4c79d640 1 -- 172.21.15.102:0/2700854962 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd44068920 msgr2=0x7fdd44068cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:37.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.362+0000 7fdd4c79d640 1 --2- 172.21.15.102:0/2700854962 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd44068920 0x7fdd44068cf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdd34009910 tx=0x7fdd340091d0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.362+0000 7fdd4c79d640 1 -- 172.21.15.102:0/2700854962 shutdown_connections 2024-09-13T17:43:37.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.362+0000 7fdd4c79d640 1 --2- 172.21.15.102:0/2700854962 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdd44069230 0x7fdd44136720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.362+0000 7fdd4c79d640 1 --2- 172.21.15.102:0/2700854962 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd44068920 0x7fdd44068cf0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.362+0000 7fdd4c79d640 1 --2- 172.21.15.102:0/2700854962 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdd4412f8d0 0x7fdd4412fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.362+0000 7fdd4c79d640 1 -- 172.21.15.102:0/2700854962 >> 172.21.15.102:0/2700854962 conn(0x7fdd440a3420 msgr2=0x7fdd4406e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:37.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.362+0000 7fdd4c79d640 1 -- 172.21.15.102:0/2700854962 shutdown_connections 2024-09-13T17:43:37.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.362+0000 7fdd4c79d640 1 -- 172.21.15.102:0/2700854962 wait complete. 2024-09-13T17:43:37.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.363+0000 7fdd4c79d640 1 Processor -- start 2024-09-13T17:43:37.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.363+0000 7fdd4c79d640 1 -- start start 2024-09-13T17:43:37.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.363+0000 7fdd4c79d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdd44068920 0x7fdd4413ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:37.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.363+0000 7fdd4c79d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdd44069230 0x7fdd4413f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:37.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.363+0000 7fdd4c79d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd4412f8d0 0x7fdd44119060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:37.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.363+0000 7fdd4c79d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4411ab00 con 0x7fdd4412f8d0 2024-09-13T17:43:37.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.363+0000 7fdd4c79d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4411b9d0 con 0x7fdd44069230 2024-09-13T17:43:37.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.363+0000 7fdd4c79d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4411bf70 con 0x7fdd44068920 2024-09-13T17:43:37.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.363+0000 7fdd4a539640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdd44068920 0x7fdd4413ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:37.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.363+0000 7fdd49d38640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdd44069230 0x7fdd4413f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:37.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.363+0000 7fdd4a539640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdd44068920 0x7fdd4413ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47424/0 (socket says 172.21.15.102:47424) 2024-09-13T17:43:37.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.363+0000 7fdd4ad3a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd4412f8d0 0x7fdd44119060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:37.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.363+0000 7fdd4a539640 1 -- 172.21.15.102:0/446700239 learned_addr learned my addr 172.21.15.102:0/446700239 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:37.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.363+0000 7fdd4a539640 1 -- 172.21.15.102:0/446700239 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdd44069230 msgr2=0x7fdd4413f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:37.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.363+0000 7fdd4a539640 1 --2- 172.21.15.102:0/446700239 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdd44069230 0x7fdd4413f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.363+0000 7fdd4a539640 1 -- 172.21.15.102:0/446700239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd4412f8d0 msgr2=0x7fdd44119060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:37.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.363+0000 7fdd4a539640 1 --2- 172.21.15.102:0/446700239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd4412f8d0 0x7fdd44119060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.363+0000 7fdd4a539640 1 -- 172.21.15.102:0/446700239 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd3c009a20 con 0x7fdd44068920 2024-09-13T17:43:37.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.363+0000 7fdd4ad3a640 1 --2- 172.21.15.102:0/446700239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd4412f8d0 0x7fdd44119060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:37.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.364+0000 7fdd4a539640 1 --2- 172.21.15.102:0/446700239 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdd44068920 0x7fdd4413ee00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdd4000ecb0 tx=0x7fdd400090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:37.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.364+0000 7fdd3a7fc640 1 -- 172.21.15.102:0/446700239 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4000cbc0 con 0x7fdd44068920 2024-09-13T17:43:37.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.364+0000 7fdd3a7fc640 1 -- 172.21.15.102:0/446700239 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4000cd20 con 0x7fdd44068920 2024-09-13T17:43:37.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.364+0000 7fdd4c79d640 1 -- 172.21.15.102:0/446700239 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd40009390 con 0x7fdd44068920 2024-09-13T17:43:37.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.364+0000 7fdd3a7fc640 1 -- 172.21.15.102:0/446700239 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd40013680 con 0x7fdd44068920 2024-09-13T17:43:37.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.364+0000 7fdd4c79d640 1 -- 172.21.15.102:0/446700239 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd34009450 con 0x7fdd44068920 2024-09-13T17:43:37.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.364+0000 7fdd3a7fc640 1 -- 172.21.15.102:0/446700239 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fdd400139f0 con 0x7fdd44068920 2024-09-13T17:43:37.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.365+0000 7fdd3a7fc640 1 -- 172.21.15.102:0/446700239 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7fdd40042580 con 0x7fdd44068920 2024-09-13T17:43:37.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.365+0000 7fdd4c79d640 1 -- 172.21.15.102:0/446700239 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd44064000 con 0x7fdd44068920 2024-09-13T17:43:37.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.369+0000 7fdd3a7fc640 1 -- 172.21.15.102:0/446700239 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fdd40013ca0 con 0x7fdd44068920 2024-09-13T17:43:37.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.524+0000 7fdd4c79d640 1 -- 172.21.15.102:0/446700239 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdd440a21a0 con 0x7fdd44068920 2024-09-13T17:43:37.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.524+0000 7fdd3a7fc640 1 -- 172.21.15.102:0/446700239 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdd400232f0 con 0x7fdd44068920 2024-09-13T17:43:37.528 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-09-13T17:43:37.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.526+0000 7fdd4c79d640 1 -- 172.21.15.102:0/446700239 >> v1:172.21.15.102:6816/34937 conn(0x7fdd28031160 legacy=0x7fdd28033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:37.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.526+0000 7fdd4c79d640 1 -- 172.21.15.102:0/446700239 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdd44068920 msgr2=0x7fdd4413ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:37.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.526+0000 7fdd4c79d640 1 --2- 172.21.15.102:0/446700239 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdd44068920 0x7fdd4413ee00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdd4000ecb0 tx=0x7fdd400090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.526+0000 7fdd4c79d640 1 -- 172.21.15.102:0/446700239 shutdown_connections 2024-09-13T17:43:37.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.526+0000 7fdd4c79d640 1 --2- 172.21.15.102:0/446700239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd4412f8d0 0x7fdd44119060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.526+0000 7fdd4c79d640 1 --2- 172.21.15.102:0/446700239 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdd44069230 0x7fdd4413f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.526+0000 7fdd4c79d640 1 --2- 172.21.15.102:0/446700239 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdd44068920 0x7fdd4413ee00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.526+0000 7fdd4c79d640 1 -- 172.21.15.102:0/446700239 >> 172.21.15.102:0/446700239 conn(0x7fdd440a3420 msgr2=0x7fdd44132780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:37.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.526+0000 7fdd4c79d640 1 -- 172.21.15.102:0/446700239 shutdown_connections 2024-09-13T17:43:37.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.526+0000 7fdd4c79d640 1 -- 172.21.15.102:0/446700239 wait complete. 2024-09-13T17:43:37.538 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:43:37.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.632+0000 7f487b71d640 1 Processor -- start 2024-09-13T17:43:37.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.632+0000 7f487b71d640 1 -- start start 2024-09-13T17:43:37.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.633+0000 7f487b71d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4874097530 0x7f4874097900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:37.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.633+0000 7f487b71d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4874097e40 0x7f4874098210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:37.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.633+0000 7f487b71d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f487412ff10 0x7f48741322f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:37.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.633+0000 7f487b71d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f487406ed00 con 0x7f4874097530 2024-09-13T17:43:37.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.633+0000 7f487b71d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4874068bd0 con 0x7f4874097e40 2024-09-13T17:43:37.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.633+0000 7f487b71d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4874098c50 con 0x7f487412ff10 2024-09-13T17:43:37.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.633+0000 7f48794b9640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4874097530 0x7f4874097900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:37.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.633+0000 7f4878cb8640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4874097e40 0x7f4874098210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:37.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.633+0000 7f4879cba640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f487412ff10 0x7f48741322f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:37.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.633+0000 7f48794b9640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4874097530 0x7f4874097900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57708/0 (socket says 172.21.15.102:57708) 2024-09-13T17:43:37.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.633+0000 7f48794b9640 1 -- 172.21.15.102:0/3948876148 learned_addr learned my addr 172.21.15.102:0/3948876148 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:37.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.633+0000 7f4878cb8640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4874097e40 0x7f4874098210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40004/0 (socket says 172.21.15.102:40004) 2024-09-13T17:43:37.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.633+0000 7f48794b9640 1 -- 172.21.15.102:0/3948876148 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f487412ff10 msgr2=0x7f48741322f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:37.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.633+0000 7f48794b9640 1 --2- 172.21.15.102:0/3948876148 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f487412ff10 0x7f48741322f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.633+0000 7f48794b9640 1 -- 172.21.15.102:0/3948876148 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4874097e40 msgr2=0x7f4874098210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:37.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.633+0000 7f48794b9640 1 --2- 172.21.15.102:0/3948876148 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4874097e40 0x7f4874098210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.633+0000 7f48794b9640 1 -- 172.21.15.102:0/3948876148 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4874099040 con 0x7f4874097530 2024-09-13T17:43:37.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.633+0000 7f4878cb8640 1 --2- 172.21.15.102:0/3948876148 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4874097e40 0x7f4874098210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:37.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.634+0000 7f48794b9640 1 --2- 172.21.15.102:0/3948876148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4874097530 0x7f4874097900 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4868009910 tx=0x7f48680091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8358012680095ae6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:37.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.634+0000 7f4867fff640 1 -- 172.21.15.102:0/3948876148 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486802ac90 con 0x7f4874097530 2024-09-13T17:43:37.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.634+0000 7f4867fff640 1 -- 172.21.15.102:0/3948876148 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f486802adf0 con 0x7f4874097530 2024-09-13T17:43:37.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.634+0000 7f4867fff640 1 -- 172.21.15.102:0/3948876148 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4868032670 con 0x7f4874097530 2024-09-13T17:43:37.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.634+0000 7f487b71d640 1 -- 172.21.15.102:0/3948876148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4874097530 msgr2=0x7f4874097900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:37.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.634+0000 7f487b71d640 1 --2- 172.21.15.102:0/3948876148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4874097530 0x7f4874097900 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4868009910 tx=0x7f48680091d0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.634+0000 7f487b71d640 1 -- 172.21.15.102:0/3948876148 shutdown_connections 2024-09-13T17:43:37.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.634+0000 7f487b71d640 1 --2- 172.21.15.102:0/3948876148 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f487412ff10 0x7f48741322f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.634+0000 7f487b71d640 1 --2- 172.21.15.102:0/3948876148 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4874097e40 0x7f4874098210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.634+0000 7f487b71d640 1 --2- 172.21.15.102:0/3948876148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4874097530 0x7f4874097900 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.634+0000 7f487b71d640 1 -- 172.21.15.102:0/3948876148 >> 172.21.15.102:0/3948876148 conn(0x7f487409e720 msgr2=0x7f487409eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:37.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.635+0000 7f487b71d640 1 -- 172.21.15.102:0/3948876148 shutdown_connections 2024-09-13T17:43:37.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.635+0000 7f487b71d640 1 -- 172.21.15.102:0/3948876148 wait complete. 2024-09-13T17:43:37.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.635+0000 7f487b71d640 1 Processor -- start 2024-09-13T17:43:37.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.635+0000 7f487b71d640 1 -- start start 2024-09-13T17:43:37.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.636+0000 7f487b71d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4874097530 0x7f4874118a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:37.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.636+0000 7f487b71d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4874097e40 0x7f4874118fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:37.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.636+0000 7f487b71d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f487412ff10 0x7f48741194e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:37.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.636+0000 7f487b71d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487411bf40 con 0x7f487412ff10 2024-09-13T17:43:37.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.636+0000 7f487b71d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f487411c950 con 0x7f4874097530 2024-09-13T17:43:37.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.636+0000 7f487b71d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f487409dee0 con 0x7f4874097e40 2024-09-13T17:43:37.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.636+0000 7f48794b9640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4874097530 0x7f4874118a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:37.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.636+0000 7f48794b9640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4874097530 0x7f4874118a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40006/0 (socket says 172.21.15.102:40006) 2024-09-13T17:43:37.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.636+0000 7f4878cb8640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4874097e40 0x7f4874118fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:37.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.636+0000 7f48794b9640 1 -- 172.21.15.102:0/1827520939 learned_addr learned my addr 172.21.15.102:0/1827520939 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:37.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.636+0000 7f4879cba640 1 --2- 172.21.15.102:0/1827520939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f487412ff10 0x7f48741194e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:37.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.636+0000 7f4878cb8640 1 -- 172.21.15.102:0/1827520939 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4874097530 msgr2=0x7f4874118a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:37.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.636+0000 7f4878cb8640 1 --2- 172.21.15.102:0/1827520939 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4874097530 0x7f4874118a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.636+0000 7f4878cb8640 1 -- 172.21.15.102:0/1827520939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f487412ff10 msgr2=0x7f48741194e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:37.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.636+0000 7f4878cb8640 1 --2- 172.21.15.102:0/1827520939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f487412ff10 0x7f48741194e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.636+0000 7f4878cb8640 1 -- 172.21.15.102:0/1827520939 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4870009390 con 0x7f4874097e40 2024-09-13T17:43:37.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.636+0000 7f48794b9640 1 --2- 172.21.15.102:0/1827520939 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4874097530 0x7f4874118a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:37.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.636+0000 7f4879cba640 1 --2- 172.21.15.102:0/1827520939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f487412ff10 0x7f48741194e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:37.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.636+0000 7f4878cb8640 1 --2- 172.21.15.102:0/1827520939 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4874097e40 0x7f4874118fa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f487000ea50 tx=0x7f48700090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:37.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.637+0000 7f48657fa640 1 -- 172.21.15.102:0/1827520939 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487000c9b0 con 0x7f4874097e40 2024-09-13T17:43:37.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.637+0000 7f48657fa640 1 -- 172.21.15.102:0/1827520939 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f487000cb10 con 0x7f4874097e40 2024-09-13T17:43:37.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.637+0000 7f48657fa640 1 -- 172.21.15.102:0/1827520939 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4870013470 con 0x7f4874097e40 2024-09-13T17:43:37.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.637+0000 7f487b71d640 1 -- 172.21.15.102:0/1827520939 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4868009450 con 0x7f4874097e40 2024-09-13T17:43:37.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.637+0000 7f487b71d640 1 -- 172.21.15.102:0/1827520939 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f487411acc0 con 0x7f4874097e40 2024-09-13T17:43:37.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.637+0000 7f48657fa640 1 -- 172.21.15.102:0/1827520939 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f4870013860 con 0x7f4874097e40 2024-09-13T17:43:37.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.638+0000 7f48657fa640 1 -- 172.21.15.102:0/1827520939 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f4870042850 con 0x7f4874097e40 2024-09-13T17:43:37.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.638+0000 7f487b71d640 1 -- 172.21.15.102:0/1827520939 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4874064000 con 0x7f4874097e40 2024-09-13T17:43:37.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.642+0000 7f48657fa640 1 -- 172.21.15.102:0/1827520939 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f4870021020 con 0x7f4874097e40 2024-09-13T17:43:37.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.793+0000 7f487b71d640 1 -- 172.21.15.102:0/1827520939 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f487406cce0 con 0x7f4874097e40 2024-09-13T17:43:37.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.794+0000 7f48657fa640 1 -- 172.21.15.102:0/1827520939 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f487001bb90 con 0x7f4874097e40 2024-09-13T17:43:37.796 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-09-13T17:43:37.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.795+0000 7f487b71d640 1 -- 172.21.15.102:0/1827520939 >> v1:172.21.15.102:6816/34937 conn(0x7f4858031160 legacy=0x7f4858033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:37.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.795+0000 7f487b71d640 1 -- 172.21.15.102:0/1827520939 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4874097e40 msgr2=0x7f4874118fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:37.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.795+0000 7f487b71d640 1 --2- 172.21.15.102:0/1827520939 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4874097e40 0x7f4874118fa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f487000ea50 tx=0x7f48700090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.796+0000 7f487b71d640 1 -- 172.21.15.102:0/1827520939 shutdown_connections 2024-09-13T17:43:37.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.796+0000 7f487b71d640 1 --2- 172.21.15.102:0/1827520939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f487412ff10 0x7f48741194e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.796+0000 7f487b71d640 1 --2- 172.21.15.102:0/1827520939 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4874097e40 0x7f4874118fa0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.796+0000 7f487b71d640 1 --2- 172.21.15.102:0/1827520939 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4874097530 0x7f4874118a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.796+0000 7f487b71d640 1 -- 172.21.15.102:0/1827520939 >> 172.21.15.102:0/1827520939 conn(0x7f487409e720 msgr2=0x7f487412f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:37.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.796+0000 7f487b71d640 1 -- 172.21.15.102:0/1827520939 shutdown_connections 2024-09-13T17:43:37.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.796+0000 7f487b71d640 1 -- 172.21.15.102:0/1827520939 wait complete. 2024-09-13T17:43:37.808 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:43:37.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.899+0000 7fb8f199d640 1 Processor -- start 2024-09-13T17:43:37.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.899+0000 7fb8f199d640 1 -- start start 2024-09-13T17:43:37.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.900+0000 7fb8f199d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb8ec068920 0x7fb8ec0a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:37.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.900+0000 7fb8f199d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8ec0a1110 0x7fb8ec0a1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:37.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.900+0000 7fb8f199d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb8ec12f540 0x7fb8ec12f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:37.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.900+0000 7fb8f199d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8ec068cf0 con 0x7fb8ec12f540 2024-09-13T17:43:37.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.900+0000 7fb8f199d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec069850 con 0x7fb8ec0a1110 2024-09-13T17:43:37.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.900+0000 7fb8f199d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec0a1a60 con 0x7fb8ec068920 2024-09-13T17:43:37.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.900+0000 7fb8eaffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb8ec068920 0x7fb8ec0a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:37.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.900+0000 7fb8e27fc640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8ec0a1110 0x7fb8ec0a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:37.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.900+0000 7fb8eb7fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb8ec12f540 0x7fb8ec12f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:37.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.900+0000 7fb8e27fc640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8ec0a1110 0x7fb8ec0a1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40014/0 (socket says 172.21.15.102:40014) 2024-09-13T17:43:37.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.900+0000 7fb8eaffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb8ec068920 0x7fb8ec0a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47450/0 (socket says 172.21.15.102:47450) 2024-09-13T17:43:37.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.900+0000 7fb8e27fc640 1 -- 172.21.15.102:0/4062185178 learned_addr learned my addr 172.21.15.102:0/4062185178 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:37.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.900+0000 7fb8eb7fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb8ec12f540 0x7fb8ec12f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57720/0 (socket says 172.21.15.102:57720) 2024-09-13T17:43:37.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.900+0000 7fb8e27fc640 1 -- 172.21.15.102:0/4062185178 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb8ec068920 msgr2=0x7fb8ec0a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:37.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.900+0000 7fb8e27fc640 1 --2- 172.21.15.102:0/4062185178 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb8ec068920 0x7fb8ec0a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.900+0000 7fb8e27fc640 1 -- 172.21.15.102:0/4062185178 >> v2:172.21.15.102:3300/0 conn(0x7fb8ec12f540 msgr2=0x7fb8ec12f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:37.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.900+0000 7fb8e27fc640 1 --2- 172.21.15.102:0/4062185178 >> v2:172.21.15.102:3300/0 conn(0x7fb8ec12f540 0x7fb8ec12f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.900+0000 7fb8e27fc640 1 -- 172.21.15.102:0/4062185178 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ec069260 con 0x7fb8ec0a1110 2024-09-13T17:43:37.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.900+0000 7fb8eb7fe640 1 --2- 172.21.15.102:0/4062185178 >> v2:172.21.15.102:3300/0 conn(0x7fb8ec12f540 0x7fb8ec12f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:37.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.900+0000 7fb8eaffd640 1 --2- 172.21.15.102:0/4062185178 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb8ec068920 0x7fb8ec0a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:37.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.901+0000 7fb8e27fc640 1 --2- 172.21.15.102:0/4062185178 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8ec0a1110 0x7fb8ec0a1520 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4009910 tx=0x7fb8d40091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c46bacdbe0b8355 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:37.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.901+0000 7fb8ea7fc640 1 -- 172.21.15.102:0/4062185178 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d402ac80 con 0x7fb8ec0a1110 2024-09-13T17:43:37.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.901+0000 7fb8ea7fc640 1 -- 172.21.15.102:0/4062185178 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d402ade0 con 0x7fb8ec0a1110 2024-09-13T17:43:37.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.901+0000 7fb8ea7fc640 1 -- 172.21.15.102:0/4062185178 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4032670 con 0x7fb8ec0a1110 2024-09-13T17:43:37.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.901+0000 7fb8f199d640 1 -- 172.21.15.102:0/4062185178 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8ec0a1110 msgr2=0x7fb8ec0a1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:37.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.901+0000 7fb8f199d640 1 --2- 172.21.15.102:0/4062185178 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8ec0a1110 0x7fb8ec0a1520 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4009910 tx=0x7fb8d40091d0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.902+0000 7fb8f199d640 1 -- 172.21.15.102:0/4062185178 shutdown_connections 2024-09-13T17:43:37.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.902+0000 7fb8f199d640 1 --2- 172.21.15.102:0/4062185178 >> v2:172.21.15.102:3300/0 conn(0x7fb8ec12f540 0x7fb8ec12f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.902+0000 7fb8f199d640 1 --2- 172.21.15.102:0/4062185178 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8ec0a1110 0x7fb8ec0a1520 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.902+0000 7fb8f199d640 1 --2- 172.21.15.102:0/4062185178 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb8ec068920 0x7fb8ec0a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.902+0000 7fb8f199d640 1 -- 172.21.15.102:0/4062185178 >> 172.21.15.102:0/4062185178 conn(0x7fb8ec0a3490 msgr2=0x7fb8ec06e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:37.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.902+0000 7fb8f199d640 1 -- 172.21.15.102:0/4062185178 shutdown_connections 2024-09-13T17:43:37.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.902+0000 7fb8f199d640 1 -- 172.21.15.102:0/4062185178 wait complete. 2024-09-13T17:43:37.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.902+0000 7fb8f199d640 1 Processor -- start 2024-09-13T17:43:37.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.903+0000 7fb8f199d640 1 -- start start 2024-09-13T17:43:37.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.903+0000 7fb8f199d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8ec068920 0x7fb8ec13ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:37.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.903+0000 7fb8f199d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8ec0a1110 0x7fb8ec13f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:37.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.903+0000 7fb8f199d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb8ec12f540 0x7fb8ec119060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:37.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.903+0000 7fb8f199d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec11cbe0 con 0x7fb8ec068920 2024-09-13T17:43:37.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.903+0000 7fb8f199d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec11bc50 con 0x7fb8ec0a1110 2024-09-13T17:43:37.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.903+0000 7fb8f199d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec12ecb0 con 0x7fb8ec12f540 2024-09-13T17:43:37.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.903+0000 7fb8eaffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8ec068920 0x7fb8ec13ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:37.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.903+0000 7fb8e27fc640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8ec0a1110 0x7fb8ec13f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:37.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.903+0000 7fb8eaffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8ec068920 0x7fb8ec13ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57736/0 (socket says 172.21.15.102:57736) 2024-09-13T17:43:37.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.903+0000 7fb8eaffd640 1 -- 172.21.15.102:0/367895171 learned_addr learned my addr 172.21.15.102:0/367895171 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:37.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.903+0000 7fb8eb7fe640 1 --2- 172.21.15.102:0/367895171 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb8ec12f540 0x7fb8ec119060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:37.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.903+0000 7fb8e27fc640 1 -- 172.21.15.102:0/367895171 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb8ec12f540 msgr2=0x7fb8ec119060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:37.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.903+0000 7fb8e27fc640 1 --2- 172.21.15.102:0/367895171 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb8ec12f540 0x7fb8ec119060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.903+0000 7fb8e27fc640 1 -- 172.21.15.102:0/367895171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8ec068920 msgr2=0x7fb8ec13ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:37.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.903+0000 7fb8e27fc640 1 --2- 172.21.15.102:0/367895171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8ec068920 0x7fb8ec13ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.903+0000 7fb8e27fc640 1 -- 172.21.15.102:0/367895171 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8dc009390 con 0x7fb8ec0a1110 2024-09-13T17:43:37.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.903+0000 7fb8eaffd640 1 --2- 172.21.15.102:0/367895171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8ec068920 0x7fb8ec13ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:37.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.903+0000 7fb8e27fc640 1 --2- 172.21.15.102:0/367895171 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8ec0a1110 0x7fb8ec13f160 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec097030 tx=0x7fb8d4032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:37.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.904+0000 7fb8e37fe640 1 -- 172.21.15.102:0/367895171 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d402ac80 con 0x7fb8ec0a1110 2024-09-13T17:43:37.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.904+0000 7fb8e37fe640 1 -- 172.21.15.102:0/367895171 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d403d630 con 0x7fb8ec0a1110 2024-09-13T17:43:37.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.904+0000 7fb8f199d640 1 -- 172.21.15.102:0/367895171 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8e4009a20 con 0x7fb8ec0a1110 2024-09-13T17:43:37.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.904+0000 7fb8e37fe640 1 -- 172.21.15.102:0/367895171 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d403d9f0 con 0x7fb8ec0a1110 2024-09-13T17:43:37.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.904+0000 7fb8f199d640 1 -- 172.21.15.102:0/367895171 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d4009450 con 0x7fb8ec0a1110 2024-09-13T17:43:37.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.905+0000 7fb8e37fe640 1 -- 172.21.15.102:0/367895171 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fb8d4039040 con 0x7fb8ec0a1110 2024-09-13T17:43:37.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.905+0000 7fb8e37fe640 1 -- 172.21.15.102:0/367895171 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7fb8d4060f70 con 0x7fb8ec0a1110 2024-09-13T17:43:37.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.905+0000 7fb8f199d640 1 -- 172.21.15.102:0/367895171 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8ec064000 con 0x7fb8ec0a1110 2024-09-13T17:43:37.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:37.909+0000 7fb8e37fe640 1 -- 172.21.15.102:0/367895171 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fb8d4037070 con 0x7fb8ec0a1110 2024-09-13T17:43:38.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.090+0000 7fb8f199d640 1 -- 172.21.15.102:0/367895171 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8ec11be70 con 0x7fb8ec0a1110 2024-09-13T17:43:38.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.090+0000 7fb8e37fe640 1 -- 172.21.15.102:0/367895171 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2266 (secure 0 0 0) 0x7fb8d4037030 con 0x7fb8ec0a1110 2024-09-13T17:43:38.092 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:43:38.093 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:42:06.611970+0000","modified":"2024-09-13T17:42:09.305093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4210":{"gid":4210,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6817/3443280887","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":3443280887}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-09-13T17:43:38.093 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 5 2024-09-13T17:43:38.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.092+0000 7fb8f199d640 1 -- 172.21.15.102:0/367895171 >> v1:172.21.15.102:6816/34937 conn(0x7fb8cc031160 legacy=0x7fb8cc033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:38.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.092+0000 7fb8f199d640 1 -- 172.21.15.102:0/367895171 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8ec0a1110 msgr2=0x7fb8ec13f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:38.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.092+0000 7fb8f199d640 1 --2- 172.21.15.102:0/367895171 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8ec0a1110 0x7fb8ec13f160 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec097030 tx=0x7fb8d4032000 comp rx=0 tx=0).stop 2024-09-13T17:43:38.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.092+0000 7fb8f199d640 1 -- 172.21.15.102:0/367895171 shutdown_connections 2024-09-13T17:43:38.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.092+0000 7fb8f199d640 1 --2- 172.21.15.102:0/367895171 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb8ec12f540 0x7fb8ec119060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.092+0000 7fb8f199d640 1 --2- 172.21.15.102:0/367895171 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb8ec0a1110 0x7fb8ec13f160 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.092+0000 7fb8f199d640 1 --2- 172.21.15.102:0/367895171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8ec068920 0x7fb8ec13ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.092+0000 7fb8f199d640 1 -- 172.21.15.102:0/367895171 >> 172.21.15.102:0/367895171 conn(0x7fb8ec0a3490 msgr2=0x7fb8ec132650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:38.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.093+0000 7fb8f199d640 1 -- 172.21.15.102:0/367895171 shutdown_connections 2024-09-13T17:43:38.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.093+0000 7fb8f199d640 1 -- 172.21.15.102:0/367895171 wait complete. 2024-09-13T17:43:38.105 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:43:38.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.195+0000 7f4b6fb9d640 1 Processor -- start 2024-09-13T17:43:38.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.195+0000 7f4b6fb9d640 1 -- start start 2024-09-13T17:43:38.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.196+0000 7f4b6fb9d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4b6809abb0 0x7f4b6812c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:38.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.196+0000 7f4b6fb9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4b6812c6a0 0x7f4b6812cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:38.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.196+0000 7f4b6fb9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4b68096d80 0x7f4b68097200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:38.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.196+0000 7f4b6fb9d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b6809af80 con 0x7f4b6809abb0 2024-09-13T17:43:38.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.196+0000 7f4b6fb9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6812cff0 con 0x7f4b6812c6a0 2024-09-13T17:43:38.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.196+0000 7f4b6fb9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6806a530 con 0x7f4b68096d80 2024-09-13T17:43:38.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.196+0000 7f4b6d939640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4b6809abb0 0x7f4b6812c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:38.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.196+0000 7f4b6e13a640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4b68096d80 0x7f4b68097200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:38.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.196+0000 7f4b6d138640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4b6812c6a0 0x7f4b6812cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:38.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.196+0000 7f4b6d939640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4b6809abb0 0x7f4b6812c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57750/0 (socket says 172.21.15.102:57750) 2024-09-13T17:43:38.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.196+0000 7f4b6e13a640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4b68096d80 0x7f4b68097200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47458/0 (socket says 172.21.15.102:47458) 2024-09-13T17:43:38.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.196+0000 7f4b6d939640 1 -- 172.21.15.102:0/2621887460 learned_addr learned my addr 172.21.15.102:0/2621887460 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:38.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.196+0000 7f4b6d939640 1 -- 172.21.15.102:0/2621887460 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4b68096d80 msgr2=0x7f4b68097200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:38.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.196+0000 7f4b6d939640 1 --2- 172.21.15.102:0/2621887460 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4b68096d80 0x7f4b68097200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.196+0000 7f4b6d939640 1 -- 172.21.15.102:0/2621887460 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4b6812c6a0 msgr2=0x7f4b6812cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:38.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.196+0000 7f4b6d939640 1 --2- 172.21.15.102:0/2621887460 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4b6812c6a0 0x7f4b6812cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.196+0000 7f4b6d939640 1 -- 172.21.15.102:0/2621887460 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b68098be0 con 0x7f4b6809abb0 2024-09-13T17:43:38.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.196+0000 7f4b6d138640 1 --2- 172.21.15.102:0/2621887460 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4b6812c6a0 0x7f4b6812cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:38.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.196+0000 7f4b6e13a640 1 --2- 172.21.15.102:0/2621887460 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4b68096d80 0x7f4b68097200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:38.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.197+0000 7f4b6d939640 1 --2- 172.21.15.102:0/2621887460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b6809abb0 0x7f4b6812c160 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4b6000dd70 tx=0x7f4b6000d660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f4375b9aec2c25 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:38.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.197+0000 7f4b6c937640 1 -- 172.21.15.102:0/2621887460 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6000d2e0 con 0x7f4b6809abb0 2024-09-13T17:43:38.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.197+0000 7f4b6c937640 1 -- 172.21.15.102:0/2621887460 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b60031050 con 0x7f4b6809abb0 2024-09-13T17:43:38.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.197+0000 7f4b6c937640 1 -- 172.21.15.102:0/2621887460 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6000b680 con 0x7f4b6809abb0 2024-09-13T17:43:38.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.198+0000 7f4b6fb9d640 1 -- 172.21.15.102:0/2621887460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b6809abb0 msgr2=0x7f4b6812c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:38.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.198+0000 7f4b6fb9d640 1 --2- 172.21.15.102:0/2621887460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b6809abb0 0x7f4b6812c160 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4b6000dd70 tx=0x7f4b6000d660 comp rx=0 tx=0).stop 2024-09-13T17:43:38.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.198+0000 7f4b6fb9d640 1 -- 172.21.15.102:0/2621887460 shutdown_connections 2024-09-13T17:43:38.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.198+0000 7f4b6fb9d640 1 --2- 172.21.15.102:0/2621887460 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4b68096d80 0x7f4b68097200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.198+0000 7f4b6fb9d640 1 --2- 172.21.15.102:0/2621887460 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4b6812c6a0 0x7f4b6812cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.198+0000 7f4b6fb9d640 1 --2- 172.21.15.102:0/2621887460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b6809abb0 0x7f4b6812c160 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.198+0000 7f4b6fb9d640 1 -- 172.21.15.102:0/2621887460 >> 172.21.15.102:0/2621887460 conn(0x7f4b6809e820 msgr2=0x7f4b6809ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:38.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.198+0000 7f4b6fb9d640 1 -- 172.21.15.102:0/2621887460 shutdown_connections 2024-09-13T17:43:38.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.198+0000 7f4b6fb9d640 1 -- 172.21.15.102:0/2621887460 wait complete. 2024-09-13T17:43:38.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.199+0000 7f4b6fb9d640 1 Processor -- start 2024-09-13T17:43:38.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.199+0000 7f4b6fb9d640 1 -- start start 2024-09-13T17:43:38.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.199+0000 7f4b6fb9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b68096d80 0x7f4b68117ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:38.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.199+0000 7f4b6fb9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4b6809abb0 0x7f4b681181e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:38.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.199+0000 7f4b6fb9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4b6812c6a0 0x7f4b6813a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:38.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.199+0000 7f4b6fb9d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b68129e70 con 0x7f4b68096d80 2024-09-13T17:43:38.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.199+0000 7f4b6fb9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b68117a70 con 0x7f4b6809abb0 2024-09-13T17:43:38.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.199+0000 7f4b6fb9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4b68119620 con 0x7f4b6812c6a0 2024-09-13T17:43:38.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.199+0000 7f4b6d939640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b68096d80 0x7f4b68117ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:38.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.199+0000 7f4b6e13a640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4b6812c6a0 0x7f4b6813a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:38.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.199+0000 7f4b6d138640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4b6809abb0 0x7f4b681181e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:38.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.199+0000 7f4b6d939640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b68096d80 0x7f4b68117ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57754/0 (socket says 172.21.15.102:57754) 2024-09-13T17:43:38.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.199+0000 7f4b6e13a640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4b6812c6a0 0x7f4b6813a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47468/0 (socket says 172.21.15.102:47468) 2024-09-13T17:43:38.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.199+0000 7f4b6d939640 1 -- 172.21.15.102:0/3518302918 learned_addr learned my addr 172.21.15.102:0/3518302918 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:38.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.199+0000 7f4b6d939640 1 -- 172.21.15.102:0/3518302918 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4b6812c6a0 msgr2=0x7f4b6813a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:38.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.199+0000 7f4b6d939640 1 --2- 172.21.15.102:0/3518302918 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4b6812c6a0 0x7f4b6813a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.199+0000 7f4b6d939640 1 -- 172.21.15.102:0/3518302918 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4b6809abb0 msgr2=0x7f4b681181e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:38.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.199+0000 7f4b6d939640 1 --2- 172.21.15.102:0/3518302918 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4b6809abb0 0x7f4b681181e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.199+0000 7f4b6d939640 1 -- 172.21.15.102:0/3518302918 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b58009a20 con 0x7f4b68096d80 2024-09-13T17:43:38.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.199+0000 7f4b6e13a640 1 --2- 172.21.15.102:0/3518302918 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4b6812c6a0 0x7f4b6813a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:38.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.200+0000 7f4b6d939640 1 --2- 172.21.15.102:0/3518302918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b68096d80 0x7f4b68117ca0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4b6000ddd0 tx=0x7f4b6003a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:38.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.200+0000 7f4b65ffb640 1 -- 172.21.15.102:0/3518302918 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6000b070 con 0x7f4b68096d80 2024-09-13T17:43:38.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.200+0000 7f4b65ffb640 1 -- 172.21.15.102:0/3518302918 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b6000d2e0 con 0x7f4b68096d80 2024-09-13T17:43:38.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.200+0000 7f4b65ffb640 1 -- 172.21.15.102:0/3518302918 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6000b610 con 0x7f4b68096d80 2024-09-13T17:43:38.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.200+0000 7f4b6fb9d640 1 -- 172.21.15.102:0/3518302918 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b5c00b440 con 0x7f4b68096d80 2024-09-13T17:43:38.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.200+0000 7f4b6fb9d640 1 -- 172.21.15.102:0/3518302918 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b6000d900 con 0x7f4b68096d80 2024-09-13T17:43:38.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.201+0000 7f4b65ffb640 1 -- 172.21.15.102:0/3518302918 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f4b6003c030 con 0x7f4b68096d80 2024-09-13T17:43:38.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.201+0000 7f4b65ffb640 1 -- 172.21.15.102:0/3518302918 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f4b60061900 con 0x7f4b68096d80 2024-09-13T17:43:38.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.201+0000 7f4b6fb9d640 1 -- 172.21.15.102:0/3518302918 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b68098e00 con 0x7f4b68096d80 2024-09-13T17:43:38.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.205+0000 7f4b65ffb640 1 -- 172.21.15.102:0/3518302918 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f4b60045080 con 0x7f4b68096d80 2024-09-13T17:43:38.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.357+0000 7f4b6fb9d640 1 -- 172.21.15.102:0/3518302918 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b68064000 con 0x7f4b68096d80 2024-09-13T17:43:38.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.357+0000 7f4b65ffb640 1 -- 172.21.15.102:0/3518302918 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+11660 (secure 0 0 0) 0x7f4b6003dda0 con 0x7f4b68096d80 2024-09-13T17:43:38.359 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:43:38.360 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":23,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:42:48.385896+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:04.672265+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:05.583134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.17","mappings":[{"from":3,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]},{"pgid":"4.25","mappings":[{"from":3,"to":2}]},{"pgid":"4.33","mappings":[{"from":3,"to":0}]},{"pgid":"4.3a","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:43:38.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.359+0000 7f4b6fb9d640 1 -- 172.21.15.102:0/3518302918 >> v1:172.21.15.102:6816/34937 conn(0x7f4b54031160 legacy=0x7f4b54033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:38.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.359+0000 7f4b6fb9d640 1 -- 172.21.15.102:0/3518302918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b68096d80 msgr2=0x7f4b68117ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:38.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.359+0000 7f4b6fb9d640 1 --2- 172.21.15.102:0/3518302918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b68096d80 0x7f4b68117ca0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4b6000ddd0 tx=0x7f4b6003a000 comp rx=0 tx=0).stop 2024-09-13T17:43:38.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.359+0000 7f4b6fb9d640 1 -- 172.21.15.102:0/3518302918 shutdown_connections 2024-09-13T17:43:38.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.359+0000 7f4b6fb9d640 1 --2- 172.21.15.102:0/3518302918 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4b6812c6a0 0x7f4b6813a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.359+0000 7f4b6fb9d640 1 --2- 172.21.15.102:0/3518302918 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4b6809abb0 0x7f4b681181e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.359+0000 7f4b6fb9d640 1 --2- 172.21.15.102:0/3518302918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b68096d80 0x7f4b68117ca0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.359+0000 7f4b6fb9d640 1 -- 172.21.15.102:0/3518302918 >> 172.21.15.102:0/3518302918 conn(0x7f4b6809e820 msgr2=0x7f4b6806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:38.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.359+0000 7f4b6fb9d640 1 -- 172.21.15.102:0/3518302918 shutdown_connections 2024-09-13T17:43:38.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.359+0000 7f4b6fb9d640 1 -- 172.21.15.102:0/3518302918 wait complete. 2024-09-13T17:43:38.372 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:54.442612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T17:41:59.076782+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 16, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:42:04.672265+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:42:05.583134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:43:38.372 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:43:38.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.466+0000 7f316099d640 1 Processor -- start 2024-09-13T17:43:38.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.466+0000 7f316099d640 1 -- start start 2024-09-13T17:43:38.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.467+0000 7f316099d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f315c0689d0 0x7f315c09d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:38.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.467+0000 7f316099d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f315c09e160 0x7f315c12fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:38.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.467+0000 7f316099d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f315c096b10 con 0x7f315c09dd70 2024-09-13T17:43:38.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.467+0000 7f316099d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f315c096db0 con 0x7f315c09e160 2024-09-13T17:43:38.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.467+0000 7f316099d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f315c097730 con 0x7f315c0689d0 2024-09-13T17:43:38.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.467+0000 7f315a59c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f315c0689d0 0x7f315c09d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:38.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.467+0000 7f3159d9b640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f315c09dd70 0x7f315c12f610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:56310/0 (socket says 172.21.15.102:56310) 2024-09-13T17:43:38.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.467+0000 7f3159d9b640 1 -- 172.21.15.102:0/1296787857 learned_addr learned my addr 172.21.15.102:0/1296787857 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:38.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.467+0000 7f315ad9d640 1 --2- 172.21.15.102:0/1296787857 >> v2:172.21.15.102:3300/0 conn(0x7f315c09e160 0x7f315c12fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:38.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.467+0000 7f315a59c640 1 -- 172.21.15.102:0/1296787857 >> v2:172.21.15.102:3300/0 conn(0x7f315c09e160 msgr2=0x7f315c12fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:38.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.467+0000 7f315a59c640 1 --2- 172.21.15.102:0/1296787857 >> v2:172.21.15.102:3300/0 conn(0x7f315c09e160 0x7f315c12fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.467+0000 7f315a59c640 1 -- 172.21.15.102:0/1296787857 >> v1:172.21.15.102:6789/0 conn(0x7f315c09dd70 legacy=0x7f315c12f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:38.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.467+0000 7f315a59c640 1 -- 172.21.15.102:0/1296787857 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f315c0696e0 con 0x7f315c0689d0 2024-09-13T17:43:38.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.467+0000 7f315959a640 1 -- 172.21.15.102:0/1296787857 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4007971452 0 0) 0x7f315c096b10 con 0x7f315c09dd70 2024-09-13T17:43:38.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.467+0000 7f315ad9d640 1 --2- 172.21.15.102:0/1296787857 >> v2:172.21.15.102:3300/0 conn(0x7f315c09e160 0x7f315c12fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:38.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.468+0000 7f315a59c640 1 --2- 172.21.15.102:0/1296787857 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f315c0689d0 0x7f315c09d830 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f314c009850 tx=0x7f314c0090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1870a58bb546d931 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:38.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.468+0000 7f315959a640 1 -- 172.21.15.102:0/1296787857 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c02ac80 con 0x7f315c0689d0 2024-09-13T17:43:38.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.468+0000 7f315959a640 1 -- 172.21.15.102:0/1296787857 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f314c02ade0 con 0x7f315c0689d0 2024-09-13T17:43:38.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.468+0000 7f315959a640 1 -- 172.21.15.102:0/1296787857 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c032670 con 0x7f315c0689d0 2024-09-13T17:43:38.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.469+0000 7f316099d640 1 -- 172.21.15.102:0/1296787857 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f315c0689d0 msgr2=0x7f315c09d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:38.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.469+0000 7f316099d640 1 --2- 172.21.15.102:0/1296787857 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f315c0689d0 0x7f315c09d830 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f314c009850 tx=0x7f314c0090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.469+0000 7f316099d640 1 -- 172.21.15.102:0/1296787857 shutdown_connections 2024-09-13T17:43:38.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.469+0000 7f316099d640 1 --2- 172.21.15.102:0/1296787857 >> v2:172.21.15.102:3300/0 conn(0x7f315c09e160 0x7f315c12fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.469+0000 7f316099d640 1 --2- 172.21.15.102:0/1296787857 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f315c0689d0 0x7f315c09d830 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.469+0000 7f316099d640 1 -- 172.21.15.102:0/1296787857 >> 172.21.15.102:0/1296787857 conn(0x7f315c06f320 msgr2=0x7f315c0a2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:38.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.469+0000 7f316099d640 1 -- 172.21.15.102:0/1296787857 shutdown_connections 2024-09-13T17:43:38.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.469+0000 7f316099d640 1 -- 172.21.15.102:0/1296787857 wait complete. 2024-09-13T17:43:38.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.469+0000 7f316099d640 1 Processor -- start 2024-09-13T17:43:38.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.470+0000 7f316099d640 1 -- start start 2024-09-13T17:43:38.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.470+0000 7f316099d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f315c0689d0 0x7f315c13eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:38.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.470+0000 7f316099d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f315c09dd70 0x7f315c13f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:38.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.470+0000 7f316099d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f315c09e160 0x7f315c11e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:38.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.470+0000 7f316099d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f315c11fd00 con 0x7f315c09dd70 2024-09-13T17:43:38.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.470+0000 7f316099d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f315c121d10 con 0x7f315c09e160 2024-09-13T17:43:38.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.470+0000 7f316099d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f315c13cb80 con 0x7f315c0689d0 2024-09-13T17:43:38.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.470+0000 7f315a59c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f315c0689d0 0x7f315c13eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:38.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.470+0000 7f3159d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f315c09dd70 0x7f315c13f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:38.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.470+0000 7f315a59c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f315c0689d0 0x7f315c13eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47494/0 (socket says 172.21.15.102:47494) 2024-09-13T17:43:38.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.470+0000 7f315a59c640 1 -- 172.21.15.102:0/3713138965 learned_addr learned my addr 172.21.15.102:0/3713138965 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:38.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.470+0000 7f315ad9d640 1 --2- 172.21.15.102:0/3713138965 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f315c09e160 0x7f315c11e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:38.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.470+0000 7f315a59c640 1 -- 172.21.15.102:0/3713138965 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f315c09e160 msgr2=0x7f315c11e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:38.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.470+0000 7f315a59c640 1 --2- 172.21.15.102:0/3713138965 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f315c09e160 0x7f315c11e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.470+0000 7f315a59c640 1 -- 172.21.15.102:0/3713138965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f315c09dd70 msgr2=0x7f315c13f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:38.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.470+0000 7f315a59c640 1 --2- 172.21.15.102:0/3713138965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f315c09dd70 0x7f315c13f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.470+0000 7f315a59c640 1 -- 172.21.15.102:0/3713138965 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3154009a20 con 0x7f315c0689d0 2024-09-13T17:43:38.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.470+0000 7f315a59c640 1 --2- 172.21.15.102:0/3713138965 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f315c0689d0 0x7f315c13eda0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f314c009360 tx=0x7f314c032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:38.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.471+0000 7f314a7fc640 1 -- 172.21.15.102:0/3713138965 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c03d700 con 0x7f315c0689d0 2024-09-13T17:43:38.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.471+0000 7f314a7fc640 1 -- 172.21.15.102:0/3713138965 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f314c03d860 con 0x7f315c0689d0 2024-09-13T17:43:38.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.471+0000 7f316099d640 1 -- 172.21.15.102:0/3713138965 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f314c009390 con 0x7f315c0689d0 2024-09-13T17:43:38.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.471+0000 7f314a7fc640 1 -- 172.21.15.102:0/3713138965 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c03dc20 con 0x7f315c0689d0 2024-09-13T17:43:38.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.471+0000 7f316099d640 1 -- 172.21.15.102:0/3713138965 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f315c09ebc0 con 0x7f315c0689d0 2024-09-13T17:43:38.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.471+0000 7f314a7fc640 1 -- 172.21.15.102:0/3713138965 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f314c039040 con 0x7f315c0689d0 2024-09-13T17:43:38.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.472+0000 7f314a7fc640 1 -- 172.21.15.102:0/3713138965 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f314c061ea0 con 0x7f315c0689d0 2024-09-13T17:43:38.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.472+0000 7f316099d640 1 -- 172.21.15.102:0/3713138965 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f315c064000 con 0x7f315c0689d0 2024-09-13T17:43:38.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.476+0000 7f314a7fc640 1 -- 172.21.15.102:0/3713138965 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f314c042080 con 0x7f315c0689d0 2024-09-13T17:43:38.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.630+0000 7f316099d640 1 -- 172.21.15.102:0/3713138965 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f315c0a39c0 con 0x7f315c0689d0 2024-09-13T17:43:38.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.631+0000 7f314a7fc640 1 -- 172.21.15.102:0/3713138965 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f314c037030 con 0x7f315c0689d0 2024-09-13T17:43:38.633 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-09-13T17:43:38.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.633+0000 7f316099d640 1 -- 172.21.15.102:0/3713138965 >> v1:172.21.15.102:6816/34937 conn(0x7f313c031160 legacy=0x7f313c033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:38.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.633+0000 7f316099d640 1 -- 172.21.15.102:0/3713138965 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f315c0689d0 msgr2=0x7f315c13eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:38.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.633+0000 7f316099d640 1 --2- 172.21.15.102:0/3713138965 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f315c0689d0 0x7f315c13eda0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f314c009360 tx=0x7f314c032000 comp rx=0 tx=0).stop 2024-09-13T17:43:38.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.633+0000 7f316099d640 1 -- 172.21.15.102:0/3713138965 shutdown_connections 2024-09-13T17:43:38.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.633+0000 7f316099d640 1 --2- 172.21.15.102:0/3713138965 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f315c09e160 0x7f315c11e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.633+0000 7f316099d640 1 --2- 172.21.15.102:0/3713138965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f315c09dd70 0x7f315c13f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.633+0000 7f316099d640 1 --2- 172.21.15.102:0/3713138965 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f315c0689d0 0x7f315c13eda0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.633+0000 7f316099d640 1 -- 172.21.15.102:0/3713138965 >> 172.21.15.102:0/3713138965 conn(0x7f315c06f320 msgr2=0x7f315c09b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:38.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.633+0000 7f316099d640 1 -- 172.21.15.102:0/3713138965 shutdown_connections 2024-09-13T17:43:38.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.633+0000 7f316099d640 1 -- 172.21.15.102:0/3713138965 wait complete. 2024-09-13T17:43:38.645 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-13T17:43:38.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.740+0000 7f4d6e68b640 1 Processor -- start 2024-09-13T17:43:38.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.740+0000 7f4d6e68b640 1 -- start start 2024-09-13T17:43:38.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.740+0000 7f4d6e68b640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4d68096a70 0x7f4d680987c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:38.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.740+0000 7f4d6e68b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d68099110 0x7f4d6812c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:38.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.740+0000 7f4d6e68b640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d680999e0 con 0x7f4d68098d00 2024-09-13T17:43:38.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.740+0000 7f4d6e68b640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d68069c30 con 0x7f4d68096a70 2024-09-13T17:43:38.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.740+0000 7f4d6e68b640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6806ed00 con 0x7f4d68099110 2024-09-13T17:43:38.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.740+0000 7f4d6cc28640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d68099110 0x7f4d6812c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:38.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.740+0000 7f4d67fff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4d68096a70 0x7f4d680987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:38.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.741+0000 7f4d6cc28640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d68099110 0x7f4d6812c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47496/0 (socket says 172.21.15.102:47496) 2024-09-13T17:43:38.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.741+0000 7f4d677fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f4d68098d00 0x7f4d6812c2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:56320/0 (socket says 172.21.15.102:56320) 2024-09-13T17:43:38.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.741+0000 7f4d6cc28640 1 -- 172.21.15.102:0/35213272 learned_addr learned my addr 172.21.15.102:0/35213272 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:38.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.741+0000 7f4d67fff640 1 -- 172.21.15.102:0/35213272 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d68099110 msgr2=0x7f4d6812c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:38.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.741+0000 7f4d67fff640 1 --2- 172.21.15.102:0/35213272 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d68099110 0x7f4d6812c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.741+0000 7f4d67fff640 1 -- 172.21.15.102:0/35213272 >> v1:172.21.15.102:6789/0 conn(0x7f4d68098d00 legacy=0x7f4d6812c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:38.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.741+0000 7f4d67fff640 1 -- 172.21.15.102:0/35213272 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d680997a0 con 0x7f4d68096a70 2024-09-13T17:43:38.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.741+0000 7f4d66ffd640 1 -- 172.21.15.102:0/35213272 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1818735767 0 0) 0x7f4d680999e0 con 0x7f4d68098d00 2024-09-13T17:43:38.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.741+0000 7f4d6cc28640 1 --2- 172.21.15.102:0/35213272 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d68099110 0x7f4d6812c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:38.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.741+0000 7f4d67fff640 1 --2- 172.21.15.102:0/35213272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d68096a70 0x7f4d680987c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4d58009850 tx=0x7f4d580090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a0ffc71b6ceb1ef server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:38.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.742+0000 7f4d66ffd640 1 -- 172.21.15.102:0/35213272 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5802ad00 con 0x7f4d68096a70 2024-09-13T17:43:38.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.742+0000 7f4d66ffd640 1 -- 172.21.15.102:0/35213272 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d5802ae60 con 0x7f4d68096a70 2024-09-13T17:43:38.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.742+0000 7f4d66ffd640 1 -- 172.21.15.102:0/35213272 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d58032670 con 0x7f4d68096a70 2024-09-13T17:43:38.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.742+0000 7f4d6e68b640 1 -- 172.21.15.102:0/35213272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d68096a70 msgr2=0x7f4d680987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:38.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.742+0000 7f4d6e68b640 1 --2- 172.21.15.102:0/35213272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d68096a70 0x7f4d680987c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4d58009850 tx=0x7f4d580090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.742+0000 7f4d6e68b640 1 -- 172.21.15.102:0/35213272 shutdown_connections 2024-09-13T17:43:38.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.742+0000 7f4d6e68b640 1 --2- 172.21.15.102:0/35213272 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d68099110 0x7f4d6812c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.742+0000 7f4d6e68b640 1 --2- 172.21.15.102:0/35213272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d68096a70 0x7f4d680987c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.742+0000 7f4d6e68b640 1 -- 172.21.15.102:0/35213272 >> 172.21.15.102:0/35213272 conn(0x7f4d6809e720 msgr2=0x7f4d6809eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:38.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.742+0000 7f4d6e68b640 1 -- 172.21.15.102:0/35213272 shutdown_connections 2024-09-13T17:43:38.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.742+0000 7f4d6e68b640 1 -- 172.21.15.102:0/35213272 wait complete. 2024-09-13T17:43:38.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.743+0000 7f4d6e68b640 1 Processor -- start 2024-09-13T17:43:38.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.743+0000 7f4d6e68b640 1 -- start start 2024-09-13T17:43:38.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.744+0000 7f4d6e68b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d68096a70 0x7f4d6811d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:38.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.744+0000 7f4d6e68b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d68098d00 0x7f4d6811a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:38.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.744+0000 7f4d6e68b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d68099110 0x7f4d6811ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:38.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.744+0000 7f4d6e68b640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6812b7d0 con 0x7f4d68098d00 2024-09-13T17:43:38.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.744+0000 7f4d6e68b640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6809a410 con 0x7f4d68099110 2024-09-13T17:43:38.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.744+0000 7f4d6e68b640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6811c970 con 0x7f4d68096a70 2024-09-13T17:43:38.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.744+0000 7f4d67fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d68096a70 0x7f4d6811d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:38.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.744+0000 7f4d677fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d68098d00 0x7f4d6811a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:38.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.744+0000 7f4d67fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d68096a70 0x7f4d6811d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47502/0 (socket says 172.21.15.102:47502) 2024-09-13T17:43:38.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.744+0000 7f4d67fff640 1 -- 172.21.15.102:0/2104205635 learned_addr learned my addr 172.21.15.102:0/2104205635 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:38.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.744+0000 7f4d6cc28640 1 --2- 172.21.15.102:0/2104205635 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d68099110 0x7f4d6811ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:38.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.744+0000 7f4d67fff640 1 -- 172.21.15.102:0/2104205635 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d68099110 msgr2=0x7f4d6811ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:38.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.744+0000 7f4d67fff640 1 --2- 172.21.15.102:0/2104205635 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d68099110 0x7f4d6811ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.744+0000 7f4d67fff640 1 -- 172.21.15.102:0/2104205635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d68098d00 msgr2=0x7f4d6811a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:38.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.744+0000 7f4d67fff640 1 --2- 172.21.15.102:0/2104205635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d68098d00 0x7f4d6811a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.744+0000 7f4d67fff640 1 -- 172.21.15.102:0/2104205635 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d60009ae0 con 0x7f4d68096a70 2024-09-13T17:43:38.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.744+0000 7f4d677fe640 1 --2- 172.21.15.102:0/2104205635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d68098d00 0x7f4d6811a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:38.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.744+0000 7f4d67fff640 1 --2- 172.21.15.102:0/2104205635 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d68096a70 0x7f4d6811d410 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4d58009850 tx=0x7f4d58032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:38.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.744+0000 7f4d4ffff640 1 -- 172.21.15.102:0/2104205635 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d580316b0 con 0x7f4d68096a70 2024-09-13T17:43:38.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.744+0000 7f4d4ffff640 1 -- 172.21.15.102:0/2104205635 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d58031810 con 0x7f4d68096a70 2024-09-13T17:43:38.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.744+0000 7f4d6e68b640 1 -- 172.21.15.102:0/2104205635 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d58009390 con 0x7f4d68096a70 2024-09-13T17:43:38.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.744+0000 7f4d4ffff640 1 -- 172.21.15.102:0/2104205635 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d58031bd0 con 0x7f4d68096a70 2024-09-13T17:43:38.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.744+0000 7f4d6e68b640 1 -- 172.21.15.102:0/2104205635 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6811b380 con 0x7f4d68096a70 2024-09-13T17:43:38.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.745+0000 7f4d4ffff640 1 -- 172.21.15.102:0/2104205635 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f4d58039040 con 0x7f4d68096a70 2024-09-13T17:43:38.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.745+0000 7f4d4ffff640 1 -- 172.21.15.102:0/2104205635 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f4d58041030 con 0x7f4d68096a70 2024-09-13T17:43:38.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.745+0000 7f4d6e68b640 1 -- 172.21.15.102:0/2104205635 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d68064000 con 0x7f4d68096a70 2024-09-13T17:43:38.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.750+0000 7f4d4ffff640 1 -- 172.21.15.102:0/2104205635 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f4d5804b020 con 0x7f4d68096a70 2024-09-13T17:43:38.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.899+0000 7f4d6e68b640 1 -- 172.21.15.102:0/2104205635 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4d680676b0 con 0x7f4d68096a70 2024-09-13T17:43:38.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.900+0000 7f4d4ffff640 1 -- 172.21.15.102:0/2104205635 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4d58037070 con 0x7f4d68096a70 2024-09-13T17:43:38.902 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-09-13T17:43:38.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.901+0000 7f4d6e68b640 1 -- 172.21.15.102:0/2104205635 >> v1:172.21.15.102:6816/34937 conn(0x7f4d54031160 legacy=0x7f4d54033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:38.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.901+0000 7f4d6e68b640 1 -- 172.21.15.102:0/2104205635 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d68096a70 msgr2=0x7f4d6811d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:38.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.901+0000 7f4d6e68b640 1 --2- 172.21.15.102:0/2104205635 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d68096a70 0x7f4d6811d410 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4d58009850 tx=0x7f4d58032000 comp rx=0 tx=0).stop 2024-09-13T17:43:38.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.901+0000 7f4d6e68b640 1 -- 172.21.15.102:0/2104205635 shutdown_connections 2024-09-13T17:43:38.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.901+0000 7f4d6e68b640 1 --2- 172.21.15.102:0/2104205635 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4d68099110 0x7f4d6811ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.901+0000 7f4d6e68b640 1 --2- 172.21.15.102:0/2104205635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d68098d00 0x7f4d6811a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.901+0000 7f4d6e68b640 1 --2- 172.21.15.102:0/2104205635 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4d68096a70 0x7f4d6811d410 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.901+0000 7f4d6e68b640 1 -- 172.21.15.102:0/2104205635 >> 172.21.15.102:0/2104205635 conn(0x7f4d6809e720 msgr2=0x7f4d6812a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:38.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.902+0000 7f4d6e68b640 1 -- 172.21.15.102:0/2104205635 shutdown_connections 2024-09-13T17:43:38.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:38.902+0000 7f4d6e68b640 1 -- 172.21.15.102:0/2104205635 wait complete. 2024-09-13T17:43:38.914 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:43:39.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.008+0000 7f565f700640 1 Processor -- start 2024-09-13T17:43:39.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.008+0000 7f565f700640 1 -- start start 2024-09-13T17:43:39.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.009+0000 7f565f700640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f565812cad0 0x7f5658131410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:39.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.009+0000 7f565f700640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5658131950 0x7f5658133d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:39.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.009+0000 7f565f700640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5658099540 con 0x7f5658096a70 2024-09-13T17:43:39.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.009+0000 7f565f700640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5658096e40 con 0x7f565812cad0 2024-09-13T17:43:39.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.009+0000 7f565f700640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f565809a8e0 con 0x7f5658131950 2024-09-13T17:43:39.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.009+0000 7f565cc9b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f565812cad0 0x7f5658131410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:39.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.009+0000 7f565dc9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5658131950 0x7f5658133d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:39.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.009+0000 7f565d49c640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f5658096a70 0x7f565812c3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:56336/0 (socket says 172.21.15.102:56336) 2024-09-13T17:43:39.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.009+0000 7f565d49c640 1 -- 172.21.15.102:0/3520019501 learned_addr learned my addr 172.21.15.102:0/3520019501 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:39.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.009+0000 7f565cc9b640 1 -- 172.21.15.102:0/3520019501 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5658131950 msgr2=0x7f5658133d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:39.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.009+0000 7f564ffff640 1 -- 172.21.15.102:0/3520019501 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1125839071 0 0) 0x7f5658099540 con 0x7f5658096a70 2024-09-13T17:43:39.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.009+0000 7f565cc9b640 1 --2- 172.21.15.102:0/3520019501 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5658131950 0x7f5658133d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.009+0000 7f565cc9b640 1 -- 172.21.15.102:0/3520019501 >> v1:172.21.15.102:6789/0 conn(0x7f5658096a70 legacy=0x7f565812c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:39.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.009+0000 7f565cc9b640 1 -- 172.21.15.102:0/3520019501 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5658099760 con 0x7f565812cad0 2024-09-13T17:43:39.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.009+0000 7f565dc9d640 1 --2- 172.21.15.102:0/3520019501 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5658131950 0x7f5658133d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:39.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.010+0000 7f565cc9b640 1 --2- 172.21.15.102:0/3520019501 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f565812cad0 0x7f5658131410 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5648009880 tx=0x7f56480090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d020346d2426547b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:39.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.010+0000 7f564ffff640 1 -- 172.21.15.102:0/3520019501 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564802ace0 con 0x7f565812cad0 2024-09-13T17:43:39.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.010+0000 7f564ffff640 1 -- 172.21.15.102:0/3520019501 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564802ae40 con 0x7f565812cad0 2024-09-13T17:43:39.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.010+0000 7f564ffff640 1 -- 172.21.15.102:0/3520019501 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5648032670 con 0x7f565812cad0 2024-09-13T17:43:39.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.010+0000 7f565f700640 1 -- 172.21.15.102:0/3520019501 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f565812cad0 msgr2=0x7f5658131410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:39.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.010+0000 7f565f700640 1 --2- 172.21.15.102:0/3520019501 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f565812cad0 0x7f5658131410 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5648009880 tx=0x7f56480090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.010+0000 7f565f700640 1 -- 172.21.15.102:0/3520019501 shutdown_connections 2024-09-13T17:43:39.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.010+0000 7f565f700640 1 --2- 172.21.15.102:0/3520019501 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5658131950 0x7f5658133d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.010+0000 7f565f700640 1 --2- 172.21.15.102:0/3520019501 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f565812cad0 0x7f5658131410 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.010+0000 7f565f700640 1 -- 172.21.15.102:0/3520019501 >> 172.21.15.102:0/3520019501 conn(0x7f565809e6b0 msgr2=0x7f565809eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:39.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.011+0000 7f565f700640 1 -- 172.21.15.102:0/3520019501 shutdown_connections 2024-09-13T17:43:39.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.011+0000 7f565f700640 1 -- 172.21.15.102:0/3520019501 wait complete. 2024-09-13T17:43:39.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.011+0000 7f565f700640 1 Processor -- start 2024-09-13T17:43:39.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.011+0000 7f565f700640 1 -- start start 2024-09-13T17:43:39.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.012+0000 7f565f700640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5658096a70 0x7f565811ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:39.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.012+0000 7f565f700640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f565812cad0 0x7f565806c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:39.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.012+0000 7f565f700640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5658131950 0x7f565806c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:39.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.012+0000 7f565f700640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565806dd50 con 0x7f5658096a70 2024-09-13T17:43:39.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.012+0000 7f565f700640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5658119f80 con 0x7f565812cad0 2024-09-13T17:43:39.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.012+0000 7f565f700640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5658131430 con 0x7f5658131950 2024-09-13T17:43:39.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.012+0000 7f565d49c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5658096a70 0x7f565811ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:39.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.012+0000 7f565cc9b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f565812cad0 0x7f565806c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:39.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.012+0000 7f565dc9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5658131950 0x7f565806c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:39.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.012+0000 7f565cc9b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f565812cad0 0x7f565806c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40098/0 (socket says 172.21.15.102:40098) 2024-09-13T17:43:39.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.012+0000 7f565cc9b640 1 -- 172.21.15.102:0/1981256684 learned_addr learned my addr 172.21.15.102:0/1981256684 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:39.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.012+0000 7f565d49c640 1 -- 172.21.15.102:0/1981256684 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5658131950 msgr2=0x7f565806c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:39.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.012+0000 7f565d49c640 1 --2- 172.21.15.102:0/1981256684 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5658131950 0x7f565806c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.012+0000 7f565d49c640 1 -- 172.21.15.102:0/1981256684 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f565812cad0 msgr2=0x7f565806c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:39.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.012+0000 7f565d49c640 1 --2- 172.21.15.102:0/1981256684 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f565812cad0 0x7f565806c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.012+0000 7f565d49c640 1 -- 172.21.15.102:0/1981256684 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5650009a20 con 0x7f5658096a70 2024-09-13T17:43:39.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.012+0000 7f565dc9d640 1 --2- 172.21.15.102:0/1981256684 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5658131950 0x7f565806c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:39.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.012+0000 7f565cc9b640 1 --2- 172.21.15.102:0/1981256684 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f565812cad0 0x7f565806c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:39.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.013+0000 7f565d49c640 1 --2- 172.21.15.102:0/1981256684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5658096a70 0x7f565811ac90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5654000c00 tx=0x7f5654007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:39.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.013+0000 7f564d7fa640 1 -- 172.21.15.102:0/1981256684 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5654005aa0 con 0x7f5658096a70 2024-09-13T17:43:39.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.013+0000 7f564d7fa640 1 -- 172.21.15.102:0/1981256684 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5654005c40 con 0x7f5658096a70 2024-09-13T17:43:39.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.013+0000 7f565f700640 1 -- 172.21.15.102:0/1981256684 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5648009390 con 0x7f5658096a70 2024-09-13T17:43:39.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.013+0000 7f564d7fa640 1 -- 172.21.15.102:0/1981256684 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56540070d0 con 0x7f5658096a70 2024-09-13T17:43:39.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.013+0000 7f565f700640 1 -- 172.21.15.102:0/1981256684 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5658119200 con 0x7f5658096a70 2024-09-13T17:43:39.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.013+0000 7f564d7fa640 1 -- 172.21.15.102:0/1981256684 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f56540074c0 con 0x7f5658096a70 2024-09-13T17:43:39.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.014+0000 7f564d7fa640 1 -- 172.21.15.102:0/1981256684 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f5654041430 con 0x7f5658096a70 2024-09-13T17:43:39.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.014+0000 7f565f700640 1 -- 172.21.15.102:0/1981256684 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5658064000 con 0x7f5658096a70 2024-09-13T17:43:39.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.018+0000 7f564d7fa640 1 -- 172.21.15.102:0/1981256684 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f5654017020 con 0x7f5658096a70 2024-09-13T17:43:39.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.167+0000 7f565f700640 1 -- 172.21.15.102:0/1981256684 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f56580008d0 con 0x7f5658096a70 2024-09-13T17:43:39.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.167+0000 7f564d7fa640 1 -- 172.21.15.102:0/1981256684 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5654024070 con 0x7f5658096a70 2024-09-13T17:43:39.170 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-09-13T17:43:39.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.169+0000 7f565f700640 1 -- 172.21.15.102:0/1981256684 >> v1:172.21.15.102:6816/34937 conn(0x7f563c031190 legacy=0x7f563c033640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:39.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.169+0000 7f565f700640 1 -- 172.21.15.102:0/1981256684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5658096a70 msgr2=0x7f565811ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:39.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.169+0000 7f565f700640 1 --2- 172.21.15.102:0/1981256684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5658096a70 0x7f565811ac90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5654000c00 tx=0x7f5654007c80 comp rx=0 tx=0).stop 2024-09-13T17:43:39.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.169+0000 7f565f700640 1 -- 172.21.15.102:0/1981256684 shutdown_connections 2024-09-13T17:43:39.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.169+0000 7f565f700640 1 --2- 172.21.15.102:0/1981256684 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5658131950 0x7f565806c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.169+0000 7f565f700640 1 --2- 172.21.15.102:0/1981256684 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f565812cad0 0x7f565806c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.169+0000 7f565f700640 1 --2- 172.21.15.102:0/1981256684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5658096a70 0x7f565811ac90 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.169+0000 7f565f700640 1 -- 172.21.15.102:0/1981256684 >> 172.21.15.102:0/1981256684 conn(0x7f565809e6b0 msgr2=0x7f5658098a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:39.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.169+0000 7f565f700640 1 -- 172.21.15.102:0/1981256684 shutdown_connections 2024-09-13T17:43:39.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.169+0000 7f565f700640 1 -- 172.21.15.102:0/1981256684 wait complete. 2024-09-13T17:43:39.181 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:43:39.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.275+0000 7f5a6931d640 1 Processor -- start 2024-09-13T17:43:39.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.275+0000 7f5a6931d640 1 -- start start 2024-09-13T17:43:39.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.276+0000 7f5a6931d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5a640a03d0 0x7f5a6412d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:39.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.276+0000 7f5a6931d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5a6412dd40 0x7f5a64067840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:39.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.276+0000 7f5a6931d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a64099cc0 con 0x7f5a6412d970 2024-09-13T17:43:39.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.276+0000 7f5a6931d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a640a07a0 con 0x7f5a6412dd40 2024-09-13T17:43:39.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.276+0000 7f5a6931d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6409e640 con 0x7f5a640a03d0 2024-09-13T17:43:39.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.276+0000 7f5a637fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5a6412dd40 0x7f5a64067840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:39.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.276+0000 7f5a62ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5a640a03d0 0x7f5a6412d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:39.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.276+0000 7f5a637fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5a6412dd40 0x7f5a64067840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57806/0 (socket says 172.21.15.102:57806) 2024-09-13T17:43:39.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.276+0000 7f5a627fc640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f5a6412d970 0x7f5a64067130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:56352/0 (socket says 172.21.15.102:56352) 2024-09-13T17:43:39.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.276+0000 7f5a637fe640 1 -- 172.21.15.102:0/39031725 learned_addr learned my addr 172.21.15.102:0/39031725 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:39.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.276+0000 7f5a62ffd640 1 -- 172.21.15.102:0/39031725 >> v2:172.21.15.102:3300/0 conn(0x7f5a6412dd40 msgr2=0x7f5a64067840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:39.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.276+0000 7f5a62ffd640 1 --2- 172.21.15.102:0/39031725 >> v2:172.21.15.102:3300/0 conn(0x7f5a6412dd40 0x7f5a64067840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.276+0000 7f5a62ffd640 1 -- 172.21.15.102:0/39031725 >> v1:172.21.15.102:6789/0 conn(0x7f5a6412d970 legacy=0x7f5a64067130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:39.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.276+0000 7f5a62ffd640 1 -- 172.21.15.102:0/39031725 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a6409e0a0 con 0x7f5a640a03d0 2024-09-13T17:43:39.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.276+0000 7f5a637fe640 1 --2- 172.21.15.102:0/39031725 >> v2:172.21.15.102:3300/0 conn(0x7f5a6412dd40 0x7f5a64067840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:39.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.277+0000 7f5a62ffd640 1 --2- 172.21.15.102:0/39031725 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5a640a03d0 0x7f5a6412d430 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5a54009e60 tx=0x7f5a54009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60d0ec72e5649c26 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:39.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.277+0000 7f5a61ffb640 1 -- 172.21.15.102:0/39031725 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a54007550 con 0x7f5a640a03d0 2024-09-13T17:43:39.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.277+0000 7f5a61ffb640 1 -- 172.21.15.102:0/39031725 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5402d050 con 0x7f5a640a03d0 2024-09-13T17:43:39.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.277+0000 7f5a61ffb640 1 -- 172.21.15.102:0/39031725 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a54007920 con 0x7f5a640a03d0 2024-09-13T17:43:39.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.278+0000 7f5a6931d640 1 -- 172.21.15.102:0/39031725 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5a640a03d0 msgr2=0x7f5a6412d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:39.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.278+0000 7f5a6931d640 1 --2- 172.21.15.102:0/39031725 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5a640a03d0 0x7f5a6412d430 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5a54009e60 tx=0x7f5a54009780 comp rx=0 tx=0).stop 2024-09-13T17:43:39.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.278+0000 7f5a6931d640 1 -- 172.21.15.102:0/39031725 shutdown_connections 2024-09-13T17:43:39.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.278+0000 7f5a6931d640 1 --2- 172.21.15.102:0/39031725 >> v2:172.21.15.102:3300/0 conn(0x7f5a6412dd40 0x7f5a64067840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.278+0000 7f5a6931d640 1 --2- 172.21.15.102:0/39031725 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5a640a03d0 0x7f5a6412d430 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.278+0000 7f5a6931d640 1 -- 172.21.15.102:0/39031725 >> 172.21.15.102:0/39031725 conn(0x7f5a6405e620 msgr2=0x7f5a6409f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:39.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.278+0000 7f5a6931d640 1 -- 172.21.15.102:0/39031725 shutdown_connections 2024-09-13T17:43:39.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.278+0000 7f5a6931d640 1 -- 172.21.15.102:0/39031725 wait complete. 2024-09-13T17:43:39.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.279+0000 7f5a6931d640 1 Processor -- start 2024-09-13T17:43:39.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.279+0000 7f5a6931d640 1 -- start start 2024-09-13T17:43:39.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.279+0000 7f5a6931d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5a640a03d0 0x7f5a64118cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:39.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.279+0000 7f5a6931d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5a6412d970 0x7f5a64119200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:39.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.279+0000 7f5a6931d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a6412dd40 0x7f5a6413c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:39.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.279+0000 7f5a6931d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6409c630 con 0x7f5a6412dd40 2024-09-13T17:43:39.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.279+0000 7f5a6931d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6411bcd0 con 0x7f5a6412d970 2024-09-13T17:43:39.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.279+0000 7f5a6931d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6409d7d0 con 0x7f5a640a03d0 2024-09-13T17:43:39.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.279+0000 7f5a62ffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5a640a03d0 0x7f5a64118cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:39.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.279+0000 7f5a627fc640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5a6412d970 0x7f5a64119200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:39.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.279+0000 7f5a637fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a6412dd40 0x7f5a6413c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:39.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.279+0000 7f5a62ffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5a640a03d0 0x7f5a64118cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47536/0 (socket says 172.21.15.102:47536) 2024-09-13T17:43:39.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.279+0000 7f5a627fc640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5a6412d970 0x7f5a64119200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40118/0 (socket says 172.21.15.102:40118) 2024-09-13T17:43:39.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.279+0000 7f5a62ffd640 1 -- 172.21.15.102:0/1215429413 learned_addr learned my addr 172.21.15.102:0/1215429413 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:39.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.279+0000 7f5a637fe640 1 -- 172.21.15.102:0/1215429413 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5a640a03d0 msgr2=0x7f5a64118cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:39.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.279+0000 7f5a637fe640 1 --2- 172.21.15.102:0/1215429413 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5a640a03d0 0x7f5a64118cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.279+0000 7f5a637fe640 1 -- 172.21.15.102:0/1215429413 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5a6412d970 msgr2=0x7f5a64119200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:39.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.279+0000 7f5a637fe640 1 --2- 172.21.15.102:0/1215429413 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5a6412d970 0x7f5a64119200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.279+0000 7f5a637fe640 1 -- 172.21.15.102:0/1215429413 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5c009450 con 0x7f5a6412dd40 2024-09-13T17:43:39.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.280+0000 7f5a637fe640 1 --2- 172.21.15.102:0/1215429413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a6412dd40 0x7f5a6413c950 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c00e9b0 tx=0x7f5a5c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:39.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.280+0000 7f5a5affd640 1 -- 172.21.15.102:0/1215429413 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c00ca10 con 0x7f5a6412dd40 2024-09-13T17:43:39.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.280+0000 7f5a5affd640 1 -- 172.21.15.102:0/1215429413 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5c00cb70 con 0x7f5a6412dd40 2024-09-13T17:43:39.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.280+0000 7f5a6931d640 1 -- 172.21.15.102:0/1215429413 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a54009a20 con 0x7f5a6412dd40 2024-09-13T17:43:39.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.280+0000 7f5a5affd640 1 -- 172.21.15.102:0/1215429413 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c013470 con 0x7f5a6412dd40 2024-09-13T17:43:39.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.280+0000 7f5a6931d640 1 -- 172.21.15.102:0/1215429413 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a6411c6e0 con 0x7f5a6412dd40 2024-09-13T17:43:39.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.281+0000 7f5a5affd640 1 -- 172.21.15.102:0/1215429413 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f5a5c0137e0 con 0x7f5a6412dd40 2024-09-13T17:43:39.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.281+0000 7f5a5affd640 1 -- 172.21.15.102:0/1215429413 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f5a5c042490 con 0x7f5a6412dd40 2024-09-13T17:43:39.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.281+0000 7f5a6931d640 1 -- 172.21.15.102:0/1215429413 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a64064600 con 0x7f5a6412dd40 2024-09-13T17:43:39.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.285+0000 7f5a5affd640 1 -- 172.21.15.102:0/1215429413 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f5a5c023ab0 con 0x7f5a6412dd40 2024-09-13T17:43:39.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.437+0000 7f5a6931d640 1 -- 172.21.15.102:0/1215429413 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5a6411b970 con 0x7f5a6412dd40 2024-09-13T17:43:39.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.437+0000 7f5a5affd640 1 -- 172.21.15.102:0/1215429413 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5a5c0232f0 con 0x7f5a6412dd40 2024-09-13T17:43:39.439 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-09-13T17:43:39.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.439+0000 7f5a6931d640 1 -- 172.21.15.102:0/1215429413 >> v1:172.21.15.102:6816/34937 conn(0x7f5a44031160 legacy=0x7f5a44033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:39.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.439+0000 7f5a6931d640 1 -- 172.21.15.102:0/1215429413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a6412dd40 msgr2=0x7f5a6413c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:39.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.439+0000 7f5a6931d640 1 --2- 172.21.15.102:0/1215429413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a6412dd40 0x7f5a6413c950 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c00e9b0 tx=0x7f5a5c0091d0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.439+0000 7f5a6931d640 1 -- 172.21.15.102:0/1215429413 shutdown_connections 2024-09-13T17:43:39.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.439+0000 7f5a6931d640 1 --2- 172.21.15.102:0/1215429413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a6412dd40 0x7f5a6413c950 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.439+0000 7f5a6931d640 1 --2- 172.21.15.102:0/1215429413 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5a6412d970 0x7f5a64119200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.439+0000 7f5a6931d640 1 --2- 172.21.15.102:0/1215429413 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5a640a03d0 0x7f5a64118cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.439+0000 7f5a6931d640 1 -- 172.21.15.102:0/1215429413 >> 172.21.15.102:0/1215429413 conn(0x7f5a6405e620 msgr2=0x7f5a641304b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:39.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.439+0000 7f5a6931d640 1 -- 172.21.15.102:0/1215429413 shutdown_connections 2024-09-13T17:43:39.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.439+0000 7f5a6931d640 1 -- 172.21.15.102:0/1215429413 wait complete. 2024-09-13T17:43:39.451 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:43:39.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.543+0000 7f059475d640 1 Processor -- start 2024-09-13T17:43:39.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.543+0000 7f059475d640 1 -- start start 2024-09-13T17:43:39.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.544+0000 7f059475d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f058c068920 0x7f058c09d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:39.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.544+0000 7f059475d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f058c09dc50 0x7f058c09e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:39.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.544+0000 7f059475d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f058c0695a0 con 0x7f058c12f310 2024-09-13T17:43:39.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.544+0000 7f059475d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f058c068cf0 con 0x7f058c068920 2024-09-13T17:43:39.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.544+0000 7f059475d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f058c09f090 con 0x7f058c09dc50 2024-09-13T17:43:39.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.544+0000 7f0591cf8640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f058c09dc50 0x7f058c09e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:39.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.544+0000 7f05924f9640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f058c068920 0x7f058c09d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:39.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.544+0000 7f05924f9640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f058c068920 0x7f058c09d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57810/0 (socket says 172.21.15.102:57810) 2024-09-13T17:43:39.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.544+0000 7f0591cf8640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f058c09dc50 0x7f058c09e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40130/0 (socket says 172.21.15.102:40130) 2024-09-13T17:43:39.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.544+0000 7f05924f9640 1 -- 172.21.15.102:0/11043457 learned_addr learned my addr 172.21.15.102:0/11043457 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:39.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.544+0000 7f0592cfa640 1 --1- 172.21.15.102:0/11043457 >> v1:172.21.15.102:6789/0 conn(0x7f058c12f310 0x7f058c12f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:56366/0 (socket says 172.21.15.102:56366) 2024-09-13T17:43:39.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.544+0000 7f05924f9640 1 -- 172.21.15.102:0/11043457 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f058c09dc50 msgr2=0x7f058c09e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:39.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.544+0000 7f05924f9640 1 --2- 172.21.15.102:0/11043457 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f058c09dc50 0x7f058c09e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.544+0000 7f05924f9640 1 -- 172.21.15.102:0/11043457 >> v1:172.21.15.102:6789/0 conn(0x7f058c12f310 legacy=0x7f058c12f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:39.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.544+0000 7f05924f9640 1 -- 172.21.15.102:0/11043457 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f058c09cce0 con 0x7f058c068920 2024-09-13T17:43:39.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.544+0000 7f05914f7640 1 -- 172.21.15.102:0/11043457 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2503167374 0 0) 0x7f058c0695a0 con 0x7f058c12f310 2024-09-13T17:43:39.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.544+0000 7f0591cf8640 1 --2- 172.21.15.102:0/11043457 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f058c09dc50 0x7f058c09e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:39.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.545+0000 7f05924f9640 1 --2- 172.21.15.102:0/11043457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f058c068920 0x7f058c09d710 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0588009910 tx=0x7f05880091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1381dcb2977978c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:39.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.545+0000 7f05914f7640 1 -- 172.21.15.102:0/11043457 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058802add0 con 0x7f058c068920 2024-09-13T17:43:39.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.545+0000 7f05914f7640 1 -- 172.21.15.102:0/11043457 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0588032440 con 0x7f058c068920 2024-09-13T17:43:39.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.545+0000 7f05914f7640 1 -- 172.21.15.102:0/11043457 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05880327e0 con 0x7f058c068920 2024-09-13T17:43:39.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.546+0000 7f059475d640 1 -- 172.21.15.102:0/11043457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f058c068920 msgr2=0x7f058c09d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:39.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.546+0000 7f059475d640 1 --2- 172.21.15.102:0/11043457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f058c068920 0x7f058c09d710 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0588009910 tx=0x7f05880091d0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.546+0000 7f059475d640 1 -- 172.21.15.102:0/11043457 shutdown_connections 2024-09-13T17:43:39.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.546+0000 7f059475d640 1 --2- 172.21.15.102:0/11043457 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f058c09dc50 0x7f058c09e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.546+0000 7f059475d640 1 --2- 172.21.15.102:0/11043457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f058c068920 0x7f058c09d710 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.546+0000 7f059475d640 1 -- 172.21.15.102:0/11043457 >> 172.21.15.102:0/11043457 conn(0x7f058c06f320 msgr2=0x7f058c0a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:39.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.546+0000 7f059475d640 1 -- 172.21.15.102:0/11043457 shutdown_connections 2024-09-13T17:43:39.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.546+0000 7f059475d640 1 -- 172.21.15.102:0/11043457 wait complete. 2024-09-13T17:43:39.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.547+0000 7f059475d640 1 Processor -- start 2024-09-13T17:43:39.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.547+0000 7f059475d640 1 -- start start 2024-09-13T17:43:39.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.547+0000 7f059475d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f058c068920 0x7f058c13ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:39.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.547+0000 7f059475d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f058c09dc50 0x7f058c13f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:39.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.547+0000 7f059475d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f058c12f310 0x7f058c11a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:39.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.547+0000 7f059475d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058c11cbc0 con 0x7f058c068920 2024-09-13T17:43:39.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.547+0000 7f059475d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f058c09fde0 con 0x7f058c09dc50 2024-09-13T17:43:39.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.547+0000 7f059475d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f058c09e620 con 0x7f058c12f310 2024-09-13T17:43:39.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.547+0000 7f05924f9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f058c068920 0x7f058c13ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:39.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.547+0000 7f0591cf8640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f058c09dc50 0x7f058c13f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:39.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.547+0000 7f05924f9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f058c068920 0x7f058c13ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57820/0 (socket says 172.21.15.102:57820) 2024-09-13T17:43:39.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.547+0000 7f05924f9640 1 -- 172.21.15.102:0/277082676 learned_addr learned my addr 172.21.15.102:0/277082676 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:39.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.547+0000 7f0592cfa640 1 --2- 172.21.15.102:0/277082676 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f058c12f310 0x7f058c11a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:39.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.547+0000 7f05924f9640 1 -- 172.21.15.102:0/277082676 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f058c12f310 msgr2=0x7f058c11a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:39.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.547+0000 7f05924f9640 1 --2- 172.21.15.102:0/277082676 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f058c12f310 0x7f058c11a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.547+0000 7f05924f9640 1 -- 172.21.15.102:0/277082676 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f058c09dc50 msgr2=0x7f058c13f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:39.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.547+0000 7f05924f9640 1 --2- 172.21.15.102:0/277082676 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f058c09dc50 0x7f058c13f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.547+0000 7f05924f9640 1 -- 172.21.15.102:0/277082676 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f057c009390 con 0x7f058c068920 2024-09-13T17:43:39.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.547+0000 7f0591cf8640 1 --2- 172.21.15.102:0/277082676 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f058c09dc50 0x7f058c13f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:39.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.548+0000 7f05924f9640 1 --2- 172.21.15.102:0/277082676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f058c068920 0x7f058c13ec90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f058802a700 tx=0x7f0588032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:39.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.548+0000 7f05827fc640 1 -- 172.21.15.102:0/277082676 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0588031bb0 con 0x7f058c068920 2024-09-13T17:43:39.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.548+0000 7f05827fc640 1 -- 172.21.15.102:0/277082676 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0588031d30 con 0x7f058c068920 2024-09-13T17:43:39.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.548+0000 7f059475d640 1 -- 172.21.15.102:0/277082676 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0588009450 con 0x7f058c068920 2024-09-13T17:43:39.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.548+0000 7f05827fc640 1 -- 172.21.15.102:0/277082676 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0588031ea0 con 0x7f058c068920 2024-09-13T17:43:39.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.548+0000 7f059475d640 1 -- 172.21.15.102:0/277082676 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f058c11c190 con 0x7f058c068920 2024-09-13T17:43:39.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.548+0000 7f05827fc640 1 -- 172.21.15.102:0/277082676 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f0588039040 con 0x7f058c068920 2024-09-13T17:43:39.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.549+0000 7f05827fc640 1 -- 172.21.15.102:0/277082676 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f05880622a0 con 0x7f058c068920 2024-09-13T17:43:39.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.549+0000 7f059475d640 1 -- 172.21.15.102:0/277082676 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f058c064000 con 0x7f058c068920 2024-09-13T17:43:39.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.553+0000 7f05827fc640 1 -- 172.21.15.102:0/277082676 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f0588039520 con 0x7f058c068920 2024-09-13T17:43:39.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.735+0000 7f059475d640 1 -- 172.21.15.102:0/277082676 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f058c09c7d0 con 0x7f058c068920 2024-09-13T17:43:39.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.735+0000 7f05827fc640 1 -- 172.21.15.102:0/277082676 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2266 (secure 0 0 0) 0x7f0588037020 con 0x7f058c068920 2024-09-13T17:43:39.737 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:43:39.737 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:42:06.611970+0000","modified":"2024-09-13T17:42:09.305093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4210":{"gid":4210,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6817/3443280887","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":3443280887}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-09-13T17:43:39.738 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 5 2024-09-13T17:43:39.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.737+0000 7f059475d640 1 -- 172.21.15.102:0/277082676 >> v1:172.21.15.102:6816/34937 conn(0x7f0570031160 legacy=0x7f0570033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:39.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.737+0000 7f059475d640 1 -- 172.21.15.102:0/277082676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f058c068920 msgr2=0x7f058c13ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:39.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.737+0000 7f059475d640 1 --2- 172.21.15.102:0/277082676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f058c068920 0x7f058c13ec90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f058802a700 tx=0x7f0588032000 comp rx=0 tx=0).stop 2024-09-13T17:43:39.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.737+0000 7f059475d640 1 -- 172.21.15.102:0/277082676 shutdown_connections 2024-09-13T17:43:39.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.737+0000 7f059475d640 1 --2- 172.21.15.102:0/277082676 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f058c12f310 0x7f058c11a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.737+0000 7f059475d640 1 --2- 172.21.15.102:0/277082676 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f058c09dc50 0x7f058c13f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.737+0000 7f059475d640 1 --2- 172.21.15.102:0/277082676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f058c068920 0x7f058c13ec90 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.737+0000 7f059475d640 1 -- 172.21.15.102:0/277082676 >> 172.21.15.102:0/277082676 conn(0x7f058c06f320 msgr2=0x7f058c0a1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:39.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.737+0000 7f059475d640 1 -- 172.21.15.102:0/277082676 shutdown_connections 2024-09-13T17:43:39.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.737+0000 7f059475d640 1 -- 172.21.15.102:0/277082676 wait complete. 2024-09-13T17:43:39.749 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:43:39.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.841+0000 7f96292e8640 1 Processor -- start 2024-09-13T17:43:39.843 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.841+0000 7f96292e8640 1 -- start start 2024-09-13T17:43:39.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.841+0000 7f96292e8640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96241305a0 0x7f9624132980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:39.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.841+0000 7f96292e8640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9624132ec0 0x7f96241352a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:39.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.841+0000 7f96292e8640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9624099ec0 con 0x7f962412d590 2024-09-13T17:43:39.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.841+0000 7f96292e8640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f962412d390 con 0x7f96241305a0 2024-09-13T17:43:39.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.841+0000 7f96292e8640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f962409bc30 con 0x7f9624132ec0 2024-09-13T17:43:39.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.841+0000 7f961bfff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96241305a0 0x7f9624132980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:39.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.841+0000 7f9622ffd640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f962412d590 0x7f962412d960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:56380/0 (socket says 172.21.15.102:56380) 2024-09-13T17:43:39.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.841+0000 7f9622ffd640 1 -- 172.21.15.102:0/474830354 learned_addr learned my addr 172.21.15.102:0/474830354 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:39.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.841+0000 7f96237fe640 1 --2- 172.21.15.102:0/474830354 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9624132ec0 0x7f96241352a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:39.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.842+0000 7f961bfff640 1 -- 172.21.15.102:0/474830354 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9624132ec0 msgr2=0x7f96241352a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:39.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.842+0000 7f961bfff640 1 --2- 172.21.15.102:0/474830354 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9624132ec0 0x7f96241352a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.842+0000 7f961bfff640 1 -- 172.21.15.102:0/474830354 >> v1:172.21.15.102:6789/0 conn(0x7f962412d590 legacy=0x7f962412d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:39.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.842+0000 7f96227fc640 1 -- 172.21.15.102:0/474830354 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3264399567 0 0) 0x7f9624099ec0 con 0x7f962412d590 2024-09-13T17:43:39.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.842+0000 7f961bfff640 1 -- 172.21.15.102:0/474830354 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f962409aab0 con 0x7f96241305a0 2024-09-13T17:43:39.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.842+0000 7f96237fe640 1 --2- 172.21.15.102:0/474830354 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9624132ec0 0x7f96241352a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:39.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.842+0000 7f961bfff640 1 --2- 172.21.15.102:0/474830354 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96241305a0 0x7f9624132980 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f960c009880 tx=0x7f960c0090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29b7ae6d0d5be74 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:39.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.842+0000 7f96227fc640 1 -- 172.21.15.102:0/474830354 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c02acd0 con 0x7f96241305a0 2024-09-13T17:43:39.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.842+0000 7f96227fc640 1 -- 172.21.15.102:0/474830354 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960c02ae30 con 0x7f96241305a0 2024-09-13T17:43:39.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.842+0000 7f96227fc640 1 -- 172.21.15.102:0/474830354 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c032670 con 0x7f96241305a0 2024-09-13T17:43:39.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.843+0000 7f96292e8640 1 -- 172.21.15.102:0/474830354 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96241305a0 msgr2=0x7f9624132980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:39.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.843+0000 7f96292e8640 1 --2- 172.21.15.102:0/474830354 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96241305a0 0x7f9624132980 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f960c009880 tx=0x7f960c0090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.843+0000 7f96292e8640 1 -- 172.21.15.102:0/474830354 shutdown_connections 2024-09-13T17:43:39.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.843+0000 7f96292e8640 1 --2- 172.21.15.102:0/474830354 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9624132ec0 0x7f96241352a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.843+0000 7f96292e8640 1 --2- 172.21.15.102:0/474830354 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96241305a0 0x7f9624132980 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.843+0000 7f96292e8640 1 -- 172.21.15.102:0/474830354 >> 172.21.15.102:0/474830354 conn(0x7f962405db50 msgr2=0x7f962405df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:39.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.843+0000 7f96292e8640 1 -- 172.21.15.102:0/474830354 shutdown_connections 2024-09-13T17:43:39.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.843+0000 7f96292e8640 1 -- 172.21.15.102:0/474830354 wait complete. 2024-09-13T17:43:39.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.844+0000 7f96292e8640 1 Processor -- start 2024-09-13T17:43:39.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.844+0000 7f96292e8640 1 -- start start 2024-09-13T17:43:39.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.844+0000 7f96292e8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f962412d590 0x7f962412ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:39.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.844+0000 7f96292e8640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96241305a0 0x7f9624116ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:39.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.844+0000 7f96292e8640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9624132ec0 0x7f9624117000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:39.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.844+0000 7f96292e8640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9624118ae0 con 0x7f962412d590 2024-09-13T17:43:39.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.844+0000 7f96292e8640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9624118720 con 0x7f96241305a0 2024-09-13T17:43:39.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.844+0000 7f96292e8640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f96241194b0 con 0x7f9624132ec0 2024-09-13T17:43:39.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.844+0000 7f9622ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f962412d590 0x7f962412ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:39.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.844+0000 7f961bfff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96241305a0 0x7f9624116ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:39.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.844+0000 7f9622ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f962412d590 0x7f962412ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57824/0 (socket says 172.21.15.102:57824) 2024-09-13T17:43:39.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.844+0000 7f9622ffd640 1 -- 172.21.15.102:0/593105288 learned_addr learned my addr 172.21.15.102:0/593105288 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:39.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.844+0000 7f96237fe640 1 --2- 172.21.15.102:0/593105288 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9624132ec0 0x7f9624117000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:39.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.845+0000 7f961bfff640 1 -- 172.21.15.102:0/593105288 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9624132ec0 msgr2=0x7f9624117000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:39.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.845+0000 7f961bfff640 1 --2- 172.21.15.102:0/593105288 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9624132ec0 0x7f9624117000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.845+0000 7f961bfff640 1 -- 172.21.15.102:0/593105288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f962412d590 msgr2=0x7f962412ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:39.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.845+0000 7f961bfff640 1 --2- 172.21.15.102:0/593105288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f962412d590 0x7f962412ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.845+0000 7f961bfff640 1 -- 172.21.15.102:0/593105288 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f961c009a20 con 0x7f96241305a0 2024-09-13T17:43:39.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.845+0000 7f96237fe640 1 --2- 172.21.15.102:0/593105288 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9624132ec0 0x7f9624117000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:39.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.845+0000 7f9622ffd640 1 --2- 172.21.15.102:0/593105288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f962412d590 0x7f962412ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:39.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.845+0000 7f961bfff640 1 --2- 172.21.15.102:0/593105288 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96241305a0 0x7f9624116ac0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f960c031000 tx=0x7f960c032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:39.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.845+0000 7f961affd640 1 -- 172.21.15.102:0/593105288 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c031700 con 0x7f96241305a0 2024-09-13T17:43:39.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.845+0000 7f961affd640 1 -- 172.21.15.102:0/593105288 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960c031860 con 0x7f96241305a0 2024-09-13T17:43:39.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.845+0000 7f96292e8640 1 -- 172.21.15.102:0/593105288 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f960c009390 con 0x7f96241305a0 2024-09-13T17:43:39.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.845+0000 7f961affd640 1 -- 172.21.15.102:0/593105288 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c031c60 con 0x7f96241305a0 2024-09-13T17:43:39.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.845+0000 7f96292e8640 1 -- 172.21.15.102:0/593105288 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9624118d20 con 0x7f96241305a0 2024-09-13T17:43:39.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.846+0000 7f961affd640 1 -- 172.21.15.102:0/593105288 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f960c039040 con 0x7f96241305a0 2024-09-13T17:43:39.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.846+0000 7f961affd640 1 -- 172.21.15.102:0/593105288 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f960c0624e0 con 0x7f96241305a0 2024-09-13T17:43:39.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.846+0000 7f96292e8640 1 -- 172.21.15.102:0/593105288 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f962409a220 con 0x7f96241305a0 2024-09-13T17:43:39.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:39.850+0000 7f961affd640 1 -- 172.21.15.102:0/593105288 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f960c048080 con 0x7f96241305a0 2024-09-13T17:43:40.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.005+0000 7f96292e8640 1 -- 172.21.15.102:0/593105288 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96240681e0 con 0x7f96241305a0 2024-09-13T17:43:40.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.006+0000 7f961affd640 1 -- 172.21.15.102:0/593105288 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+11660 (secure 0 0 0) 0x7f960c037020 con 0x7f96241305a0 2024-09-13T17:43:40.007 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:43:40.008 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":23,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:42:48.385896+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:04.672265+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:05.583134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.17","mappings":[{"from":3,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]},{"pgid":"4.25","mappings":[{"from":3,"to":2}]},{"pgid":"4.33","mappings":[{"from":3,"to":0}]},{"pgid":"4.3a","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:43:40.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.007+0000 7f96292e8640 1 -- 172.21.15.102:0/593105288 >> v1:172.21.15.102:6816/34937 conn(0x7f9604031160 legacy=0x7f9604033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.008+0000 7f96292e8640 1 -- 172.21.15.102:0/593105288 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96241305a0 msgr2=0x7f9624116ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.008+0000 7f96292e8640 1 --2- 172.21.15.102:0/593105288 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96241305a0 0x7f9624116ac0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f960c031000 tx=0x7f960c032000 comp rx=0 tx=0).stop 2024-09-13T17:43:40.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.008+0000 7f96292e8640 1 -- 172.21.15.102:0/593105288 shutdown_connections 2024-09-13T17:43:40.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.008+0000 7f96292e8640 1 --2- 172.21.15.102:0/593105288 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9624132ec0 0x7f9624117000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.008+0000 7f96292e8640 1 --2- 172.21.15.102:0/593105288 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f96241305a0 0x7f9624116ac0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.008+0000 7f96292e8640 1 --2- 172.21.15.102:0/593105288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f962412d590 0x7f962412ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.008+0000 7f96292e8640 1 -- 172.21.15.102:0/593105288 >> 172.21.15.102:0/593105288 conn(0x7f962405db50 msgr2=0x7f9624132080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:40.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.008+0000 7f96292e8640 1 -- 172.21.15.102:0/593105288 shutdown_connections 2024-09-13T17:43:40.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.008+0000 7f96292e8640 1 -- 172.21.15.102:0/593105288 wait complete. 2024-09-13T17:43:40.021 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:43:40.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.111+0000 7fdd73d9d640 1 Processor -- start 2024-09-13T17:43:40.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.111+0000 7fdd73d9d640 1 -- start start 2024-09-13T17:43:40.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.112+0000 7fdd73d9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdd6c068920 0x7fdd6c09d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.112+0000 7fdd73d9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdd6c09dc50 0x7fdd6c09e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.112+0000 7fdd73d9d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fdd6c12f310 0x7fdd6c12f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.112+0000 7fdd73d9d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd6c068cf0 con 0x7fdd6c12f310 2024-09-13T17:43:40.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.112+0000 7fdd73d9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c0695a0 con 0x7fdd6c09dc50 2024-09-13T17:43:40.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.112+0000 7fdd73d9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c09f090 con 0x7fdd6c068920 2024-09-13T17:43:40.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.112+0000 7fdd71338640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdd6c09dc50 0x7fdd6c09e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.112+0000 7fdd71b39640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdd6c068920 0x7fdd6c09d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.112+0000 7fdd7233a640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fdd6c12f310 0x7fdd6c12f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.112+0000 7fdd71b39640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdd6c068920 0x7fdd6c09d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47572/0 (socket says 172.21.15.102:47572) 2024-09-13T17:43:40.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.112+0000 7fdd71338640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdd6c09dc50 0x7fdd6c09e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:40158/0 (socket says 172.21.15.102:40158) 2024-09-13T17:43:40.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.112+0000 7fdd71b39640 1 -- 172.21.15.102:0/445282737 learned_addr learned my addr 172.21.15.102:0/445282737 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:40.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.112+0000 7fdd71338640 1 -- 172.21.15.102:0/445282737 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdd6c068920 msgr2=0x7fdd6c09d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:40.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.112+0000 7fdd71338640 1 --2- 172.21.15.102:0/445282737 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdd6c068920 0x7fdd6c09d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.112+0000 7fdd71338640 1 -- 172.21.15.102:0/445282737 >> v2:172.21.15.102:3300/0 conn(0x7fdd6c12f310 msgr2=0x7fdd6c12f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:40.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.112+0000 7fdd71338640 1 --2- 172.21.15.102:0/445282737 >> v2:172.21.15.102:3300/0 conn(0x7fdd6c12f310 0x7fdd6c12f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.112+0000 7fdd71338640 1 -- 172.21.15.102:0/445282737 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c09cce0 con 0x7fdd6c09dc50 2024-09-13T17:43:40.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.112+0000 7fdd7233a640 1 --2- 172.21.15.102:0/445282737 >> v2:172.21.15.102:3300/0 conn(0x7fdd6c12f310 0x7fdd6c12f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:40.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.113+0000 7fdd71338640 1 --2- 172.21.15.102:0/445282737 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdd6c09dc50 0x7fdd6c09e020 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c009850 tx=0x7fdd5c0090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24fb551ccd34d8f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:40.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.113+0000 7fdd70b37640 1 -- 172.21.15.102:0/445282737 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c02ac50 con 0x7fdd6c09dc50 2024-09-13T17:43:40.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.113+0000 7fdd70b37640 1 -- 172.21.15.102:0/445282737 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5c02adb0 con 0x7fdd6c09dc50 2024-09-13T17:43:40.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.113+0000 7fdd70b37640 1 -- 172.21.15.102:0/445282737 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c032670 con 0x7fdd6c09dc50 2024-09-13T17:43:40.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.114+0000 7fdd73d9d640 1 -- 172.21.15.102:0/445282737 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdd6c09dc50 msgr2=0x7fdd6c09e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.114+0000 7fdd73d9d640 1 --2- 172.21.15.102:0/445282737 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdd6c09dc50 0x7fdd6c09e020 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c009850 tx=0x7fdd5c0090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.114+0000 7fdd73d9d640 1 -- 172.21.15.102:0/445282737 shutdown_connections 2024-09-13T17:43:40.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.114+0000 7fdd73d9d640 1 --2- 172.21.15.102:0/445282737 >> v2:172.21.15.102:3300/0 conn(0x7fdd6c12f310 0x7fdd6c12f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.114+0000 7fdd73d9d640 1 --2- 172.21.15.102:0/445282737 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdd6c09dc50 0x7fdd6c09e020 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.114+0000 7fdd73d9d640 1 --2- 172.21.15.102:0/445282737 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdd6c068920 0x7fdd6c09d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.114+0000 7fdd73d9d640 1 -- 172.21.15.102:0/445282737 >> 172.21.15.102:0/445282737 conn(0x7fdd6c06f320 msgr2=0x7fdd6c0a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:40.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.114+0000 7fdd73d9d640 1 -- 172.21.15.102:0/445282737 shutdown_connections 2024-09-13T17:43:40.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.114+0000 7fdd73d9d640 1 -- 172.21.15.102:0/445282737 wait complete. 2024-09-13T17:43:40.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.114+0000 7fdd73d9d640 1 Processor -- start 2024-09-13T17:43:40.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.115+0000 7fdd73d9d640 1 -- start start 2024-09-13T17:43:40.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.115+0000 7fdd73d9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdd6c068920 0x7fdd6c12ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.115+0000 7fdd73d9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdd6c09dc50 0x7fdd6c11abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.115+0000 7fdd73d9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd6c12f310 0x7fdd6c11b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.115+0000 7fdd73d9d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c11cb40 con 0x7fdd6c12f310 2024-09-13T17:43:40.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.115+0000 7fdd73d9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c11d580 con 0x7fdd6c068920 2024-09-13T17:43:40.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.115+0000 7fdd73d9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c11c600 con 0x7fdd6c09dc50 2024-09-13T17:43:40.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.115+0000 7fdd71b39640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdd6c068920 0x7fdd6c12ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.115+0000 7fdd71338640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdd6c09dc50 0x7fdd6c11abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.115+0000 7fdd7233a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd6c12f310 0x7fdd6c11b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.115+0000 7fdd71338640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdd6c09dc50 0x7fdd6c11abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47574/0 (socket says 172.21.15.102:47574) 2024-09-13T17:43:40.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.115+0000 7fdd71338640 1 -- 172.21.15.102:0/2577148278 learned_addr learned my addr 172.21.15.102:0/2577148278 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:40.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.115+0000 7fdd71338640 1 -- 172.21.15.102:0/2577148278 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdd6c068920 msgr2=0x7fdd6c12ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.115+0000 7fdd71338640 1 --2- 172.21.15.102:0/2577148278 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdd6c068920 0x7fdd6c12ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.115+0000 7fdd71338640 1 -- 172.21.15.102:0/2577148278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd6c12f310 msgr2=0x7fdd6c11b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.115+0000 7fdd71338640 1 --2- 172.21.15.102:0/2577148278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd6c12f310 0x7fdd6c11b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.115+0000 7fdd71338640 1 -- 172.21.15.102:0/2577148278 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd64009a20 con 0x7fdd6c09dc50 2024-09-13T17:43:40.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.115+0000 7fdd7233a640 1 --2- 172.21.15.102:0/2577148278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd6c12f310 0x7fdd6c11b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:40.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.116+0000 7fdd71338640 1 --2- 172.21.15.102:0/2577148278 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdd6c09dc50 0x7fdd6c11abb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c02a580 tx=0x7fdd5c032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:40.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.116+0000 7fdd61ffb640 1 -- 172.21.15.102:0/2577148278 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c03d720 con 0x7fdd6c09dc50 2024-09-13T17:43:40.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.116+0000 7fdd61ffb640 1 -- 172.21.15.102:0/2577148278 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5c03d880 con 0x7fdd6c09dc50 2024-09-13T17:43:40.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.116+0000 7fdd73d9d640 1 -- 172.21.15.102:0/2577148278 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd5c009390 con 0x7fdd6c09dc50 2024-09-13T17:43:40.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.116+0000 7fdd61ffb640 1 -- 172.21.15.102:0/2577148278 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c03dc20 con 0x7fdd6c09dc50 2024-09-13T17:43:40.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.116+0000 7fdd73d9d640 1 -- 172.21.15.102:0/2577148278 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd6c11cd80 con 0x7fdd6c09dc50 2024-09-13T17:43:40.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.117+0000 7fdd61ffb640 1 -- 172.21.15.102:0/2577148278 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fdd5c039040 con 0x7fdd6c09dc50 2024-09-13T17:43:40.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.117+0000 7fdd61ffb640 1 -- 172.21.15.102:0/2577148278 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7fdd5c061d90 con 0x7fdd6c09dc50 2024-09-13T17:43:40.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.117+0000 7fdd73d9d640 1 -- 172.21.15.102:0/2577148278 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd6c064000 con 0x7fdd6c09dc50 2024-09-13T17:43:40.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.121+0000 7fdd61ffb640 1 -- 172.21.15.102:0/2577148278 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fdd5c037020 con 0x7fdd6c09dc50 2024-09-13T17:43:40.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.302+0000 7fdd73d9d640 1 -- 172.21.15.102:0/2577148278 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd6c09e650 con 0x7fdd6c09dc50 2024-09-13T17:43:40.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.303+0000 7fdd61ffb640 1 -- 172.21.15.102:0/2577148278 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2266 (secure 0 0 0) 0x7fdd5c042070 con 0x7fdd6c09dc50 2024-09-13T17:43:40.305 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:43:40.306 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:42:06.611970+0000","modified":"2024-09-13T17:42:09.305093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4210":{"gid":4210,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6817/3443280887","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":3443280887}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-09-13T17:43:40.306 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 5 2024-09-13T17:43:40.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.305+0000 7fdd73d9d640 1 -- 172.21.15.102:0/2577148278 >> v1:172.21.15.102:6816/34937 conn(0x7fdd50031160 legacy=0x7fdd50033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.305+0000 7fdd73d9d640 1 -- 172.21.15.102:0/2577148278 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdd6c09dc50 msgr2=0x7fdd6c11abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.305+0000 7fdd73d9d640 1 --2- 172.21.15.102:0/2577148278 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdd6c09dc50 0x7fdd6c11abb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c02a580 tx=0x7fdd5c032000 comp rx=0 tx=0).stop 2024-09-13T17:43:40.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.305+0000 7fdd73d9d640 1 -- 172.21.15.102:0/2577148278 shutdown_connections 2024-09-13T17:43:40.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.305+0000 7fdd73d9d640 1 --2- 172.21.15.102:0/2577148278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd6c12f310 0x7fdd6c11b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.305+0000 7fdd73d9d640 1 --2- 172.21.15.102:0/2577148278 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdd6c09dc50 0x7fdd6c11abb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.305+0000 7fdd73d9d640 1 --2- 172.21.15.102:0/2577148278 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdd6c068920 0x7fdd6c12ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.305+0000 7fdd73d9d640 1 -- 172.21.15.102:0/2577148278 >> 172.21.15.102:0/2577148278 conn(0x7fdd6c06f320 msgr2=0x7fdd6c0a1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:40.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.305+0000 7fdd73d9d640 1 -- 172.21.15.102:0/2577148278 shutdown_connections 2024-09-13T17:43:40.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.305+0000 7fdd73d9d640 1 -- 172.21.15.102:0/2577148278 wait complete. 2024-09-13T17:43:40.417 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T17:42:06.611970+0000', 'modified': '2024-09-13T17:42:09.305093+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4210}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4210': {'gid': 4210, 'name': 'a', 'rank': 0, 'incarnation': 3, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.102:6817/3443280887', 'addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.102:6817', 'nonce': 3443280887}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-09-13T17:43:40.418 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T17:43:40.418 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-13T17:43:40.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.513+0000 7fe025554640 1 Processor -- start 2024-09-13T17:43:40.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.513+0000 7fe025554640 1 -- start start 2024-09-13T17:43:40.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.514+0000 7fe025554640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe02012cc00 0x7fe020131430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.514+0000 7fe025554640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe020131970 0x7fe020133d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.514+0000 7fe025554640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe020097d00 con 0x7fe02009c1e0 2024-09-13T17:43:40.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.514+0000 7fe025554640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe02009c5b0 con 0x7fe020131970 2024-09-13T17:43:40.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.514+0000 7fe025554640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe0200680c0 con 0x7fe02012cc00 2024-09-13T17:43:40.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.514+0000 7fe01e7fc640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe02012cc00 0x7fe020131430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.514+0000 7fe01effd640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fe02009c1e0 0x7fe02012c4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:56386/0 (socket says 172.21.15.102:56386) 2024-09-13T17:43:40.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.514+0000 7fe01f7fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe020131970 0x7fe020133d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.514+0000 7fe01effd640 1 -- 172.21.15.102:0/1238477862 learned_addr learned my addr 172.21.15.102:0/1238477862 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:40.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.514+0000 7fe01e7fc640 1 -- 172.21.15.102:0/1238477862 >> v2:172.21.15.102:3300/0 conn(0x7fe020131970 msgr2=0x7fe020133d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:40.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.514+0000 7fe01e7fc640 1 --2- 172.21.15.102:0/1238477862 >> v2:172.21.15.102:3300/0 conn(0x7fe020131970 0x7fe020133d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.514+0000 7fe01e7fc640 1 -- 172.21.15.102:0/1238477862 >> v1:172.21.15.102:6789/0 conn(0x7fe02009c1e0 legacy=0x7fe02012c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.514+0000 7fe01e7fc640 1 -- 172.21.15.102:0/1238477862 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe020097a50 con 0x7fe02012cc00 2024-09-13T17:43:40.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.515+0000 7fe01e7fc640 1 --2- 172.21.15.102:0/1238477862 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe02012cc00 0x7fe020131430 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe014009850 tx=0x7fe0140090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab98d347ce4d665f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:40.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.515+0000 7fe01dffb640 1 -- 172.21.15.102:0/1238477862 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01402ad00 con 0x7fe02012cc00 2024-09-13T17:43:40.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.515+0000 7fe01dffb640 1 -- 172.21.15.102:0/1238477862 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01402ae60 con 0x7fe02012cc00 2024-09-13T17:43:40.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.515+0000 7fe01dffb640 1 -- 172.21.15.102:0/1238477862 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe014032670 con 0x7fe02012cc00 2024-09-13T17:43:40.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.515+0000 7fe025554640 1 -- 172.21.15.102:0/1238477862 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe02012cc00 msgr2=0x7fe020131430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.515+0000 7fe025554640 1 --2- 172.21.15.102:0/1238477862 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe02012cc00 0x7fe020131430 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe014009850 tx=0x7fe0140090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.515+0000 7fe025554640 1 -- 172.21.15.102:0/1238477862 shutdown_connections 2024-09-13T17:43:40.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.515+0000 7fe025554640 1 --2- 172.21.15.102:0/1238477862 >> v2:172.21.15.102:3300/0 conn(0x7fe020131970 0x7fe020133d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.515+0000 7fe025554640 1 --2- 172.21.15.102:0/1238477862 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe02012cc00 0x7fe020131430 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.516+0000 7fe025554640 1 -- 172.21.15.102:0/1238477862 >> 172.21.15.102:0/1238477862 conn(0x7fe02009f8b0 msgr2=0x7fe02009b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:40.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.516+0000 7fe025554640 1 -- 172.21.15.102:0/1238477862 shutdown_connections 2024-09-13T17:43:40.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.516+0000 7fe025554640 1 -- 172.21.15.102:0/1238477862 wait complete. 2024-09-13T17:43:40.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.516+0000 7fe025554640 1 Processor -- start 2024-09-13T17:43:40.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.516+0000 7fe025554640 1 -- start start 2024-09-13T17:43:40.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.517+0000 7fe025554640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe02009c1e0 0x7fe02013cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.517+0000 7fe025554640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe02012cc00 0x7fe02013d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.517+0000 7fe025554640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe020131970 0x7fe02013d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.517+0000 7fe025554640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe020127660 con 0x7fe02009c1e0 2024-09-13T17:43:40.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.517+0000 7fe025554640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe02006e9b0 con 0x7fe02012cc00 2024-09-13T17:43:40.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.517+0000 7fe025554640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe02012ba50 con 0x7fe020131970 2024-09-13T17:43:40.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.517+0000 7fe01effd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe02009c1e0 0x7fe02013cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.517+0000 7fe01effd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe02009c1e0 0x7fe02013cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57868/0 (socket says 172.21.15.102:57868) 2024-09-13T17:43:40.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.517+0000 7fe01effd640 1 -- 172.21.15.102:0/3037173164 learned_addr learned my addr 172.21.15.102:0/3037173164 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:40.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.517+0000 7fe01f7fe640 1 --2- 172.21.15.102:0/3037173164 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe020131970 0x7fe02013d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.517+0000 7fe01e7fc640 1 --2- 172.21.15.102:0/3037173164 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe02012cc00 0x7fe02013d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.517+0000 7fe01effd640 1 -- 172.21.15.102:0/3037173164 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe020131970 msgr2=0x7fe02013d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.517+0000 7fe01effd640 1 --2- 172.21.15.102:0/3037173164 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe020131970 0x7fe02013d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.517+0000 7fe01effd640 1 -- 172.21.15.102:0/3037173164 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe02012cc00 msgr2=0x7fe02013d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.517+0000 7fe01effd640 1 --2- 172.21.15.102:0/3037173164 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe02012cc00 0x7fe02013d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.517+0000 7fe01effd640 1 -- 172.21.15.102:0/3037173164 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe014009390 con 0x7fe02009c1e0 2024-09-13T17:43:40.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.517+0000 7fe01f7fe640 1 --2- 172.21.15.102:0/3037173164 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe020131970 0x7fe02013d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:40.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.517+0000 7fe01effd640 1 --2- 172.21.15.102:0/3037173164 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe02009c1e0 0x7fe02013cdf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe01000c590 tx=0x7fe010007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:40.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.518+0000 7fe006ffd640 1 -- 172.21.15.102:0/3037173164 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe010005aa0 con 0x7fe02009c1e0 2024-09-13T17:43:40.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.518+0000 7fe006ffd640 1 -- 172.21.15.102:0/3037173164 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe010005c20 con 0x7fe02009c1e0 2024-09-13T17:43:40.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.518+0000 7fe025554640 1 -- 172.21.15.102:0/3037173164 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe020126390 con 0x7fe02009c1e0 2024-09-13T17:43:40.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.518+0000 7fe006ffd640 1 -- 172.21.15.102:0/3037173164 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0100070f0 con 0x7fe02009c1e0 2024-09-13T17:43:40.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.518+0000 7fe025554640 1 -- 172.21.15.102:0/3037173164 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe020096490 con 0x7fe02009c1e0 2024-09-13T17:43:40.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.518+0000 7fe006ffd640 1 -- 172.21.15.102:0/3037173164 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fe010007500 con 0x7fe02009c1e0 2024-09-13T17:43:40.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.519+0000 7fe006ffd640 1 -- 172.21.15.102:0/3037173164 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7fe0100414b0 con 0x7fe02009c1e0 2024-09-13T17:43:40.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.520+0000 7fe0279ec740 1 Processor -- start 2024-09-13T17:43:40.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.520+0000 7fe0279ec740 1 -- start start 2024-09-13T17:43:40.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.521+0000 7fe0279ec740 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x555bf87aba90 0x555bf87abe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.521+0000 7fe0279ec740 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x555bf8791050 0x555bf87ac3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.521+0000 7fe0279ec740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf87a6510 0x555bf87a6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.521+0000 7fe0279ec740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bf86fbe70 con 0x555bf87a6510 2024-09-13T17:43:40.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.521+0000 7fe0279ec740 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x555bf86f9600 con 0x555bf8791050 2024-09-13T17:43:40.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.521+0000 7fe0279ec740 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x555bf86f52e0 con 0x555bf87aba90 2024-09-13T17:43:40.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.521+0000 7fe01effd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x555bf87aba90 0x555bf87abe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.521+0000 7fe01e7fc640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x555bf8791050 0x555bf87ac3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.521+0000 7fe01effd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x555bf87aba90 0x555bf87abe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47618/0 (socket says 172.21.15.102:47618) 2024-09-13T17:43:40.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.521+0000 7fe01effd640 1 -- 172.21.15.102:0/2848511251 learned_addr learned my addr 172.21.15.102:0/2848511251 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:40.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.521+0000 7fe01f7fe640 1 --2- 172.21.15.102:0/2848511251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf87a6510 0x555bf87a6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.521+0000 7fe01effd640 1 -- 172.21.15.102:0/2848511251 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x555bf8791050 msgr2=0x555bf87ac3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:40.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.521+0000 7fe01effd640 1 --2- 172.21.15.102:0/2848511251 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x555bf8791050 0x555bf87ac3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.521+0000 7fe01effd640 1 -- 172.21.15.102:0/2848511251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf87a6510 msgr2=0x555bf87a6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:40.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.521+0000 7fe01effd640 1 --2- 172.21.15.102:0/2848511251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf87a6510 0x555bf87a6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.521+0000 7fe01effd640 1 -- 172.21.15.102:0/2848511251 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bf841d9e0 con 0x555bf87aba90 2024-09-13T17:43:40.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.521+0000 7fe01e7fc640 1 --2- 172.21.15.102:0/2848511251 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x555bf8791050 0x555bf87ac3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:40.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.521+0000 7fe01f7fe640 1 --2- 172.21.15.102:0/2848511251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf87a6510 0x555bf87a6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:40.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.521+0000 7fe01effd640 1 --2- 172.21.15.102:0/2848511251 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x555bf87aba90 0x555bf87abe60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe010040240 tx=0x7fe010011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=348087f3f2b9e1af server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:40.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.521+0000 7fdffffff640 1 -- 172.21.15.102:0/2848511251 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe010010e90 con 0x555bf87aba90 2024-09-13T17:43:40.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.521+0000 7fdffffff640 1 -- 172.21.15.102:0/2848511251 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe010010af0 con 0x555bf87aba90 2024-09-13T17:43:40.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.521+0000 7fdffffff640 1 -- 172.21.15.102:0/2848511251 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe010005030 con 0x555bf87aba90 2024-09-13T17:43:40.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.522+0000 7fe0279ec740 1 -- 172.21.15.102:0/2848511251 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x555bf87aba90 msgr2=0x555bf87abe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.522+0000 7fe0279ec740 1 --2- 172.21.15.102:0/2848511251 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x555bf87aba90 0x555bf87abe60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe010040240 tx=0x7fe010011000 comp rx=0 tx=0).stop 2024-09-13T17:43:40.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.522+0000 7fe0279ec740 1 -- 172.21.15.102:0/2848511251 shutdown_connections 2024-09-13T17:43:40.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.522+0000 7fe0279ec740 1 --2- 172.21.15.102:0/2848511251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf87a6510 0x555bf87a6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.522+0000 7fe0279ec740 1 --2- 172.21.15.102:0/2848511251 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x555bf8791050 0x555bf87ac3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.522+0000 7fe0279ec740 1 --2- 172.21.15.102:0/2848511251 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x555bf87aba90 0x555bf87abe60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.522+0000 7fe0279ec740 1 -- 172.21.15.102:0/2848511251 >> 172.21.15.102:0/2848511251 conn(0x555bf8791b30 msgr2=0x555bf8793300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:40.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.522+0000 7fe0279ec740 1 -- 172.21.15.102:0/2848511251 shutdown_connections 2024-09-13T17:43:40.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.522+0000 7fe0279ec740 1 -- 172.21.15.102:0/2848511251 wait complete. 2024-09-13T17:43:40.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.522+0000 7fe0279ec740 1 -- start start 2024-09-13T17:43:40.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.522+0000 7fe0279ec740 1 Processor -- start 2024-09-13T17:43:40.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.523+0000 7fe0279ec740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf8791050 0x555bf87b1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.523+0000 7fe0279ec740 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x555bf87a6510 0x555bf87b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.523+0000 7fe0279ec740 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x555bf87aba90 0x555bf87b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.523+0000 7fe0279ec740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bf86971b0 con 0x555bf8791050 2024-09-13T17:43:40.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.523+0000 7fe0279ec740 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x555bf8696e50 con 0x555bf87aba90 2024-09-13T17:43:40.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.523+0000 7fe0279ec740 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x555bf843f590 con 0x555bf87a6510 2024-09-13T17:43:40.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.523+0000 7fe0279ec740 10 client.0 _ll_register_callbacks cb 0x555bf86eeb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:43:40.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.523+0000 7fe01effd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf8791050 0x555bf87b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.523+0000 7fe01e7fc640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x555bf87a6510 0x555bf87b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.523+0000 7fe01effd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf8791050 0x555bf87b1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57898/0 (socket says 172.21.15.102:57898) 2024-09-13T17:43:40.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.523+0000 7fe01effd640 1 -- 172.21.15.102:0/3165414154 learned_addr learned my addr 172.21.15.102:0/3165414154 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:40.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.523+0000 7fe01f7fe640 1 --2- 172.21.15.102:0/3165414154 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x555bf87aba90 0x555bf87b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.523+0000 7fe01effd640 1 -- 172.21.15.102:0/3165414154 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x555bf87a6510 msgr2=0x555bf87b1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:40.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.523+0000 7fe01effd640 1 --2- 172.21.15.102:0/3165414154 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x555bf87a6510 0x555bf87b1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.523+0000 7fe01effd640 1 -- 172.21.15.102:0/3165414154 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x555bf87aba90 msgr2=0x555bf87b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:40.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.523+0000 7fe01effd640 1 --2- 172.21.15.102:0/3165414154 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x555bf87aba90 0x555bf87b20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.523+0000 7fe01effd640 1 -- 172.21.15.102:0/3165414154 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe020096490 con 0x555bf8791050 2024-09-13T17:43:40.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.524+0000 7fe01effd640 1 --2- 172.21.15.102:0/3165414154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf8791050 0x555bf87b1620 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe01000e450 tx=0x7fe010032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6dc6e70284b9c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:40.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.524+0000 7fdffd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-13T17:43:40.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.524+0000 7fdffd7fa640 1 -- 172.21.15.102:0/3165414154 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe010010e90 con 0x555bf8791050 2024-09-13T17:43:40.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.524+0000 7fdffd7fa640 1 -- 172.21.15.102:0/3165414154 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7fe010005030 con 0x555bf8791050 2024-09-13T17:43:40.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.524+0000 7fe0279ec740 1 -- 172.21.15.102:0/3165414154 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555bf83fb3c0 con 0x555bf8791050 2024-09-13T17:43:40.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.524+0000 7fdffd7fa640 1 -- 172.21.15.102:0/3165414154 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=5) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe01001be70 con 0x555bf8791050 2024-09-13T17:43:40.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.524+0000 7fe0279ec740 10 client.5016 fetch_fsmap learned FSMap version 5 2024-09-13T17:43:40.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.524+0000 7fe0279ec740 1 -- 172.21.15.102:0/3165414154 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=5,monmap=2+}) v3 -- 0x555bf86a7ab0 con 0x555bf8791050 2024-09-13T17:43:40.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.524+0000 7fdffd7fa640 1 -- 172.21.15.102:0/3165414154 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555bf83fb3c0 con 0x555bf8791050 2024-09-13T17:43:40.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.524+0000 7fdffd7fa640 1 -- 172.21.15.102:0/3165414154 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 5) v1 ==== 1430+0+0 (secure 0 0 0) 0x7fe010010680 con 0x555bf8791050 2024-09-13T17:43:40.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.524+0000 7fe0279ec740 10 client.5016 fetch_fsmap finished waiting for FSMap version 5 2024-09-13T17:43:40.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.524+0000 7fe0279ec740 10 client.5016 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-13T17:43:40.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.524+0000 7fe0279ec740 20 client.5016 populate_metadata read hostname 'smithi102' 2024-09-13T17:43:40.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.524+0000 7fe0279ec740 4 client.5016 mds_command: new command op to 4210 tid=0[{"prefix": "get_command_descriptions"}] 2024-09-13T17:43:40.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.524+0000 7fe0279ec740 1 -- 172.21.15.102:0/3165414154 --> v1:172.21.15.102:6817/3443280887 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555bf840c6c0 con 0x555bf87c8620 2024-09-13T17:43:40.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.524+0000 7fdffd7fa640 10 client.5016 ms_handle_connect on v1:172.21.15.102:6817/3443280887 2024-09-13T17:43:40.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.525+0000 7fdffd7fa640 1 -- 172.21.15.102:0/3165414154 <== mds.0 v1:172.21.15.102:6817/3443280887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17536 (unknown 0 0 610023333) 0x555bf840c6c0 con 0x555bf87c8620 2024-09-13T17:43:40.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.525+0000 7fdffd7fa640 10 client.5016 handle_command_reply: tid=0 2024-09-13T17:43:40.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.525+0000 7fe0279ec740 1 client.5016 shutdown 2024-09-13T17:43:40.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.525+0000 7fe0279ec740 1 -- 172.21.15.102:0/3165414154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf8791050 msgr2=0x555bf87b1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.525+0000 7fe0279ec740 1 --2- 172.21.15.102:0/3165414154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf8791050 0x555bf87b1620 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe01000e450 tx=0x7fe010032000 comp rx=0 tx=0).stop 2024-09-13T17:43:40.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.525+0000 7fe0279ec740 1 -- 172.21.15.102:0/3165414154 shutdown_connections 2024-09-13T17:43:40.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.525+0000 7fe0279ec740 1 --2- 172.21.15.102:0/3165414154 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x555bf87aba90 0x555bf87b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.525+0000 7fdffd7fa640 0 client.5016 ms_handle_reset on v1:172.21.15.102:6817/3443280887 2024-09-13T17:43:40.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.525+0000 7fe0279ec740 1 --2- 172.21.15.102:0/3165414154 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x555bf87a6510 0x555bf87b1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.525+0000 7fe0279ec740 1 --2- 172.21.15.102:0/3165414154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf8791050 0x555bf87b1620 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.525+0000 7fe0279ec740 1 -- 172.21.15.102:0/3165414154 >> 172.21.15.102:0/3165414154 conn(0x555bf8791b30 msgr2=0x555bf87abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:40.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.525+0000 7fe0279ec740 1 -- 172.21.15.102:0/3165414154 shutdown_connections 2024-09-13T17:43:40.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.525+0000 7fe0279ec740 1 -- 172.21.15.102:0/3165414154 wait complete. 2024-09-13T17:43:40.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.525+0000 7fe0279ec740 20 client.5016 trim_cache size 0 max 16384 2024-09-13T17:43:40.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.539+0000 7fe0279ec740 1 Processor -- start 2024-09-13T17:43:40.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.540+0000 7fe0279ec740 1 -- start start 2024-09-13T17:43:40.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.540+0000 7fe0279ec740 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x555bf8791050 0x555bf87a85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.540+0000 7fe0279ec740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf87a6510 0x555bf87cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.540+0000 7fe0279ec740 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x555bf87aba90 0x555bf878ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.540+0000 7fe0279ec740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bf8790840 con 0x555bf87a6510 2024-09-13T17:43:40.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.540+0000 7fe0279ec740 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x555bf87ae200 con 0x555bf8791050 2024-09-13T17:43:40.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.540+0000 7fe0279ec740 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x555bf86965d0 con 0x555bf87aba90 2024-09-13T17:43:40.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.540+0000 7fe01e7fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf87a6510 0x555bf87cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.540+0000 7fe01f7fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x555bf87aba90 0x555bf878ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.540+0000 7fe01effd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x555bf8791050 0x555bf87a85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.540+0000 7fe01f7fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x555bf87aba90 0x555bf878ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47638/0 (socket says 172.21.15.102:47638) 2024-09-13T17:43:40.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.540+0000 7fe01e7fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf87a6510 0x555bf87cba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57914/0 (socket says 172.21.15.102:57914) 2024-09-13T17:43:40.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.540+0000 7fe01f7fe640 1 -- 172.21.15.102:0/2644667292 learned_addr learned my addr 172.21.15.102:0/2644667292 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:40.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.540+0000 7fe01f7fe640 1 -- 172.21.15.102:0/2644667292 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x555bf8791050 msgr2=0x555bf87a85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:40.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.540+0000 7fe01f7fe640 1 --2- 172.21.15.102:0/2644667292 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x555bf8791050 0x555bf87a85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.540+0000 7fe01f7fe640 1 -- 172.21.15.102:0/2644667292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf87a6510 msgr2=0x555bf87cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:40.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.540+0000 7fe01f7fe640 1 --2- 172.21.15.102:0/2644667292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf87a6510 0x555bf87cba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.540+0000 7fe01f7fe640 1 -- 172.21.15.102:0/2644667292 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bf869bbc0 con 0x555bf87aba90 2024-09-13T17:43:40.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.540+0000 7fe01effd640 1 --2- 172.21.15.102:0/2644667292 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x555bf8791050 0x555bf87a85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:40.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.540+0000 7fe01f7fe640 1 --2- 172.21.15.102:0/2644667292 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x555bf87aba90 0x555bf878ee60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe02006a0d0 tx=0x7fe01800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c295a3dad22e2ff server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:40.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.541+0000 7fe004ff9640 1 -- 172.21.15.102:0/2644667292 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe018017b80 con 0x555bf87aba90 2024-09-13T17:43:40.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.541+0000 7fe004ff9640 1 -- 172.21.15.102:0/2644667292 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01800f040 con 0x555bf87aba90 2024-09-13T17:43:40.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.541+0000 7fe004ff9640 1 -- 172.21.15.102:0/2644667292 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01801f3f0 con 0x555bf87aba90 2024-09-13T17:43:40.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.541+0000 7fe0279ec740 1 -- 172.21.15.102:0/2644667292 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x555bf87aba90 msgr2=0x555bf878ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.541+0000 7fe0279ec740 1 --2- 172.21.15.102:0/2644667292 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x555bf87aba90 0x555bf878ee60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe02006a0d0 tx=0x7fe01800b040 comp rx=0 tx=0).stop 2024-09-13T17:43:40.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.541+0000 7fe0279ec740 1 -- 172.21.15.102:0/2644667292 shutdown_connections 2024-09-13T17:43:40.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.541+0000 7fe0279ec740 1 --2- 172.21.15.102:0/2644667292 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x555bf87aba90 0x555bf878ee60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.541+0000 7fe0279ec740 1 --2- 172.21.15.102:0/2644667292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf87a6510 0x555bf87cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.541+0000 7fe0279ec740 1 --2- 172.21.15.102:0/2644667292 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x555bf8791050 0x555bf87a85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.541+0000 7fe0279ec740 1 -- 172.21.15.102:0/2644667292 >> 172.21.15.102:0/2644667292 conn(0x555bf8791b30 msgr2=0x555bf87a68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:40.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.541+0000 7fe0279ec740 1 -- 172.21.15.102:0/2644667292 shutdown_connections 2024-09-13T17:43:40.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.541+0000 7fe0279ec740 1 -- 172.21.15.102:0/2644667292 wait complete. 2024-09-13T17:43:40.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.542+0000 7fe0279ec740 1 -- start start 2024-09-13T17:43:40.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.542+0000 7fe0279ec740 1 Processor -- start 2024-09-13T17:43:40.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.542+0000 7fe0279ec740 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x555bf8791050 0x555bf8818160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.542+0000 7fe0279ec740 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x555bf87a6510 0x555bf88186a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.542+0000 7fe0279ec740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf87aba90 0x555bf87b8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.542+0000 7fe0279ec740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bf87fbf50 con 0x555bf87aba90 2024-09-13T17:43:40.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.542+0000 7fe0279ec740 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x555bf87cbcf0 con 0x555bf8791050 2024-09-13T17:43:40.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.542+0000 7fe0279ec740 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x555bf87fc840 con 0x555bf87a6510 2024-09-13T17:43:40.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.542+0000 7fe0279ec740 10 client.0 _ll_register_callbacks cb 0x7fe010015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:43:40.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.542+0000 7fe01e7fc640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x555bf87a6510 0x555bf88186a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.542+0000 7fe01f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf87aba90 0x555bf87b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.542+0000 7fe01effd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x555bf8791050 0x555bf8818160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.542+0000 7fe01e7fc640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x555bf87a6510 0x555bf88186a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47650/0 (socket says 172.21.15.102:47650) 2024-09-13T17:43:40.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.542+0000 7fe01e7fc640 1 -- 172.21.15.102:0/2058722739 learned_addr learned my addr 172.21.15.102:0/2058722739 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:40.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.543+0000 7fe01e7fc640 1 -- 172.21.15.102:0/2058722739 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x555bf8791050 msgr2=0x555bf8818160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:40.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.543+0000 7fe01e7fc640 1 --2- 172.21.15.102:0/2058722739 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x555bf8791050 0x555bf8818160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.543+0000 7fe01e7fc640 1 -- 172.21.15.102:0/2058722739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf87aba90 msgr2=0x555bf87b8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:40.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.543+0000 7fe01e7fc640 1 --2- 172.21.15.102:0/2058722739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf87aba90 0x555bf87b8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.543+0000 7fe01e7fc640 1 -- 172.21.15.102:0/2058722739 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555bf86a7ab0 con 0x555bf87a6510 2024-09-13T17:43:40.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.543+0000 7fe01f7fe640 1 --2- 172.21.15.102:0/2058722739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf87aba90 0x555bf87b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:40.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.543+0000 7fe01e7fc640 1 --2- 172.21.15.102:0/2058722739 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x555bf87a6510 0x555bf88186a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe014009130 tx=0x7fe01403f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93d652ebe8651f68 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:40.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.543+0000 7fdffd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.102:3302/0 2024-09-13T17:43:40.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.543+0000 7fdffd7fa640 1 -- 172.21.15.102:0/2058722739 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe014032cb0 con 0x555bf87a6510 2024-09-13T17:43:40.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.543+0000 7fdffd7fa640 1 -- 172.21.15.102:0/2058722739 <== mon.2 v2:172.21.15.102:3302/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7fe01404b070 con 0x555bf87a6510 2024-09-13T17:43:40.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.543+0000 7fe0279ec740 1 -- 172.21.15.102:0/2058722739 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe010010af0 con 0x555bf87a6510 2024-09-13T17:43:40.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.543+0000 7fdffd7fa640 1 -- 172.21.15.102:0/2058722739 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_get_version_reply(handle=1 version=5) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe014032e10 con 0x555bf87a6510 2024-09-13T17:43:40.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.543+0000 7fe0279ec740 10 client.4778 fetch_fsmap learned FSMap version 5 2024-09-13T17:43:40.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.543+0000 7fe0279ec740 1 -- 172.21.15.102:0/2058722739 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,fsmap=5,monmap=2+}) v3 -- 0x7fe01800e7e0 con 0x555bf87a6510 2024-09-13T17:43:40.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.543+0000 7fdffd7fa640 1 -- 172.21.15.102:0/2058722739 <== mon.2 v2:172.21.15.102:3302/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe010010af0 con 0x555bf87a6510 2024-09-13T17:43:40.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.543+0000 7fdffd7fa640 1 -- 172.21.15.102:0/2058722739 <== mon.2 v2:172.21.15.102:3302/0 5 ==== fsmap(e 5) v1 ==== 1430+0+0 (secure 0 0 0) 0x7fe01403d450 con 0x555bf87a6510 2024-09-13T17:43:40.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.543+0000 7fe0279ec740 10 client.4778 fetch_fsmap finished waiting for FSMap version 5 2024-09-13T17:43:40.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.543+0000 7fe0279ec740 10 client.4778 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-13T17:43:40.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.543+0000 7fe0279ec740 20 client.4778 populate_metadata read hostname 'smithi102' 2024-09-13T17:43:40.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.543+0000 7fe0279ec740 4 client.4778 mds_command: new command op to 4210 tid=0[{"prefix": "status"}] 2024-09-13T17:43:40.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.543+0000 7fe0279ec740 1 -- 172.21.15.102:0/2058722739 --> v1:172.21.15.102:6817/3443280887 -- command(tid 0: {"prefix": "status"}) v1 -- 0x555bf845fd10 con 0x555bf87b9140 2024-09-13T17:43:40.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.544+0000 7fdffd7fa640 10 client.4778 ms_handle_connect on v1:172.21.15.102:6817/3443280887 2024-09-13T17:43:40.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.544+0000 7fdffd7fa640 1 -- 172.21.15.102:0/2058722739 <== mds.0 v1:172.21.15.102:6817/3443280887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (unknown 0 0 348977983) 0x555bf845fd10 con 0x555bf87b9140 2024-09-13T17:43:40.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.544+0000 7fdffd7fa640 10 client.4778 handle_command_reply: tid=0 2024-09-13T17:43:40.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.544+0000 7fe0279ec740 1 client.4778 shutdown 2024-09-13T17:43:40.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.544+0000 7fe0279ec740 1 -- 172.21.15.102:0/2058722739 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x555bf87a6510 msgr2=0x555bf88186a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.544+0000 7fe0279ec740 1 --2- 172.21.15.102:0/2058722739 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x555bf87a6510 0x555bf88186a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe014009130 tx=0x7fe01403f040 comp rx=0 tx=0).stop 2024-09-13T17:43:40.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.544+0000 7fe0279ec740 1 -- 172.21.15.102:0/2058722739 shutdown_connections 2024-09-13T17:43:40.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.544+0000 7fe0279ec740 1 --2- 172.21.15.102:0/2058722739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf87aba90 0x555bf87b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.544+0000 7fe0279ec740 1 --2- 172.21.15.102:0/2058722739 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x555bf87a6510 0x555bf88186a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.544+0000 7fdffd7fa640 0 client.4778 ms_handle_reset on v1:172.21.15.102:6817/3443280887 2024-09-13T17:43:40.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.544+0000 7fe0279ec740 1 --2- 172.21.15.102:0/2058722739 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x555bf8791050 0x555bf8818160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.544+0000 7fe0279ec740 1 -- 172.21.15.102:0/2058722739 >> 172.21.15.102:0/2058722739 conn(0x555bf8791b30 msgr2=0x555bf87c4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:40.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.544+0000 7fe0279ec740 1 -- 172.21.15.102:0/2058722739 shutdown_connections 2024-09-13T17:43:40.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.544+0000 7fe0279ec740 1 -- 172.21.15.102:0/2058722739 wait complete. 2024-09-13T17:43:40.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.544+0000 7fe0279ec740 20 client.4778 trim_cache size 0 max 16384 2024-09-13T17:43:40.553 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-13T17:43:40.553 INFO:teuthology.orchestra.run.smithi102.stdout: "cluster_fsid": "6507a4f0-6c1b-42c0-9c4d-5af24a57491d", 2024-09-13T17:43:40.553 INFO:teuthology.orchestra.run.smithi102.stdout: "whoami": 0, 2024-09-13T17:43:40.553 INFO:teuthology.orchestra.run.smithi102.stdout: "id": 4210, 2024-09-13T17:43:40.553 INFO:teuthology.orchestra.run.smithi102.stdout: "want_state": "up:active", 2024-09-13T17:43:40.553 INFO:teuthology.orchestra.run.smithi102.stdout: "state": "up:active", 2024-09-13T17:43:40.554 INFO:teuthology.orchestra.run.smithi102.stdout: "fs_name": "cephfs", 2024-09-13T17:43:40.554 INFO:teuthology.orchestra.run.smithi102.stdout: "rank_uptime": 93.242978356999998, 2024-09-13T17:43:40.554 INFO:teuthology.orchestra.run.smithi102.stdout: "mdsmap_epoch": 5, 2024-09-13T17:43:40.554 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_epoch": 23, 2024-09-13T17:43:40.554 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_epoch_barrier": 20, 2024-09-13T17:43:40.554 INFO:teuthology.orchestra.run.smithi102.stdout: "uptime": 98.274961830999999 2024-09-13T17:43:40.554 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-13T17:43:40.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.545+0000 7fe025554640 1 -- 172.21.15.102:0/3037173164 >> v1:172.21.15.102:6816/34937 conn(0x7fe00c031190 legacy=0x7fe00c033640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.545+0000 7fe025554640 1 -- 172.21.15.102:0/3037173164 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe02009c1e0 msgr2=0x7fe02013cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.545+0000 7fe025554640 1 --2- 172.21.15.102:0/3037173164 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe02009c1e0 0x7fe02013cdf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe01000c590 tx=0x7fe010007ee0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.545+0000 7fe025554640 1 -- 172.21.15.102:0/3037173164 shutdown_connections 2024-09-13T17:43:40.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.545+0000 7fe025554640 1 --2- 172.21.15.102:0/3037173164 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe020131970 0x7fe02013d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.545+0000 7fe025554640 1 --2- 172.21.15.102:0/3037173164 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe02012cc00 0x7fe02013d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.545+0000 7fe025554640 1 --2- 172.21.15.102:0/3037173164 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe02009c1e0 0x7fe02013cdf0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.545+0000 7fe025554640 1 -- 172.21.15.102:0/3037173164 >> 172.21.15.102:0/3037173164 conn(0x7fe02009f8b0 msgr2=0x7fe02012a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:40.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.545+0000 7fe025554640 1 -- 172.21.15.102:0/3037173164 shutdown_connections 2024-09-13T17:43:40.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.545+0000 7fe025554640 1 -- 172.21.15.102:0/3037173164 wait complete. 2024-09-13T17:43:40.557 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-13T17:43:40.567 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-13T17:43:40.567 INFO:tasks.ceph_fuse:config is {'client.0': None} 2024-09-13T17:43:40.568 INFO:tasks.ceph_fuse:client.0 config is {} 2024-09-13T17:43:40.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:40.568 DEBUG:teuthology.orchestra.run.smithi171:> ip netns list 2024-09-13T17:43:40.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:40.588 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip link delete ceph-brx 2024-09-13T17:43:40.663 INFO:teuthology.orchestra.run.smithi171.stderr:Cannot find device "ceph-brx" 2024-09-13T17:43:40.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:43:40.665 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-13T17:43:40.665 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-13T17:43:40.665 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T17:43:40.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.757+0000 7f8ad311d640 1 Processor -- start 2024-09-13T17:43:40.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.757+0000 7f8ad311d640 1 -- start start 2024-09-13T17:43:40.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.757+0000 7f8ad311d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8acc1305a0 0x7f8acc132980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.757+0000 7f8ad311d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8acc132ec0 0x7f8acc1352a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.757+0000 7f8ad311d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8acc099ec0 con 0x7f8acc12d590 2024-09-13T17:43:40.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.757+0000 7f8ad311d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8acc12d390 con 0x7f8acc132ec0 2024-09-13T17:43:40.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.757+0000 7f8ad311d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8acc09bc30 con 0x7f8acc1305a0 2024-09-13T17:43:40.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.757+0000 7f8ad16ba640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8acc132ec0 0x7f8acc1352a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.757+0000 7f8ad0eb9640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f8acc12d590 0x7f8acc12d960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:56392/0 (socket says 172.21.15.102:56392) 2024-09-13T17:43:40.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.757+0000 7f8acbfff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8acc1305a0 0x7f8acc132980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.757+0000 7f8ad0eb9640 1 -- 172.21.15.102:0/3048841100 learned_addr learned my addr 172.21.15.102:0/3048841100 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:40.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.758+0000 7f8acb7fe640 1 -- 172.21.15.102:0/3048841100 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4164453121 0 0) 0x7f8acc099ec0 con 0x7f8acc12d590 2024-09-13T17:43:40.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.758+0000 7f8ad16ba640 1 -- 172.21.15.102:0/3048841100 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8acc1305a0 msgr2=0x7f8acc132980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:40.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.758+0000 7f8ad16ba640 1 --2- 172.21.15.102:0/3048841100 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8acc1305a0 0x7f8acc132980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.758+0000 7f8ad16ba640 1 -- 172.21.15.102:0/3048841100 >> v1:172.21.15.102:6789/0 conn(0x7f8acc12d590 legacy=0x7f8acc12d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.758+0000 7f8ad16ba640 1 -- 172.21.15.102:0/3048841100 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8acc09aab0 con 0x7f8acc132ec0 2024-09-13T17:43:40.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.758+0000 7f8acbfff640 1 --2- 172.21.15.102:0/3048841100 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8acc1305a0 0x7f8acc132980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:40.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.758+0000 7f8ad16ba640 1 --2- 172.21.15.102:0/3048841100 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8acc132ec0 0x7f8acc1352a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009880 tx=0x7f8ab40090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6a32c70eaec07e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:40.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.758+0000 7f8acb7fe640 1 -- 172.21.15.102:0/3048841100 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab402ace0 con 0x7f8acc132ec0 2024-09-13T17:43:40.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.759+0000 7f8acb7fe640 1 -- 172.21.15.102:0/3048841100 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab402ae40 con 0x7f8acc132ec0 2024-09-13T17:43:40.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.759+0000 7f8acb7fe640 1 -- 172.21.15.102:0/3048841100 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4032670 con 0x7f8acc132ec0 2024-09-13T17:43:40.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.759+0000 7f8ad311d640 1 -- 172.21.15.102:0/3048841100 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8acc132ec0 msgr2=0x7f8acc1352a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.759+0000 7f8ad311d640 1 --2- 172.21.15.102:0/3048841100 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8acc132ec0 0x7f8acc1352a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009880 tx=0x7f8ab40090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.759+0000 7f8ad311d640 1 -- 172.21.15.102:0/3048841100 shutdown_connections 2024-09-13T17:43:40.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.759+0000 7f8ad311d640 1 --2- 172.21.15.102:0/3048841100 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8acc132ec0 0x7f8acc1352a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.759+0000 7f8ad311d640 1 --2- 172.21.15.102:0/3048841100 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8acc1305a0 0x7f8acc132980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.759+0000 7f8ad311d640 1 -- 172.21.15.102:0/3048841100 >> 172.21.15.102:0/3048841100 conn(0x7f8acc05db50 msgr2=0x7f8acc05df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:40.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.759+0000 7f8ad311d640 1 -- 172.21.15.102:0/3048841100 shutdown_connections 2024-09-13T17:43:40.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.759+0000 7f8ad311d640 1 -- 172.21.15.102:0/3048841100 wait complete. 2024-09-13T17:43:40.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.760+0000 7f8ad311d640 1 Processor -- start 2024-09-13T17:43:40.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.760+0000 7f8ad311d640 1 -- start start 2024-09-13T17:43:40.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.760+0000 7f8ad311d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8acc12d590 0x7f8acc118410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.760+0000 7f8ad311d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8acc1305a0 0x7f8acc13a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.760+0000 7f8ad311d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8acc132ec0 0x7f8acc13aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.760+0000 7f8ad311d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8acc117c30 con 0x7f8acc1305a0 2024-09-13T17:43:40.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.760+0000 7f8ad311d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8acc133290 con 0x7f8acc132ec0 2024-09-13T17:43:40.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.760+0000 7f8ad311d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8acc132a20 con 0x7f8acc12d590 2024-09-13T17:43:40.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.761+0000 7f8ad0eb9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8acc12d590 0x7f8acc118410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.761+0000 7f8ad16ba640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8acc132ec0 0x7f8acc13aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.761+0000 7f8ad0eb9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8acc12d590 0x7f8acc118410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:47666/0 (socket says 172.21.15.102:47666) 2024-09-13T17:43:40.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.761+0000 7f8ad0eb9640 1 -- 172.21.15.102:0/377417843 learned_addr learned my addr 172.21.15.102:0/377417843 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:40.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.761+0000 7f8acbfff640 1 --2- 172.21.15.102:0/377417843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8acc1305a0 0x7f8acc13a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.761+0000 7f8ad0eb9640 1 -- 172.21.15.102:0/377417843 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8acc132ec0 msgr2=0x7f8acc13aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.761+0000 7f8ad0eb9640 1 --2- 172.21.15.102:0/377417843 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8acc132ec0 0x7f8acc13aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.761+0000 7f8ad0eb9640 1 -- 172.21.15.102:0/377417843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8acc1305a0 msgr2=0x7f8acc13a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.761+0000 7f8ad0eb9640 1 --2- 172.21.15.102:0/377417843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8acc1305a0 0x7f8acc13a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.761+0000 7f8ad0eb9640 1 -- 172.21.15.102:0/377417843 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8abc00de30 con 0x7f8acc12d590 2024-09-13T17:43:40.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.761+0000 7f8ad0eb9640 1 --2- 172.21.15.102:0/377417843 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8acc12d590 0x7f8acc118410 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8ac400c5e0 tx=0x7f8ac4007d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:40.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.761+0000 7f8ac8ff9640 1 -- 172.21.15.102:0/377417843 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4005aa0 con 0x7f8acc12d590 2024-09-13T17:43:40.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.761+0000 7f8ac8ff9640 1 -- 172.21.15.102:0/377417843 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac4005c20 con 0x7f8acc12d590 2024-09-13T17:43:40.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.761+0000 7f8ad311d640 1 -- 172.21.15.102:0/377417843 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab4009390 con 0x7f8acc12d590 2024-09-13T17:43:40.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.761+0000 7f8ac8ff9640 1 -- 172.21.15.102:0/377417843 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4007340 con 0x7f8acc12d590 2024-09-13T17:43:40.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.761+0000 7f8ad311d640 1 -- 172.21.15.102:0/377417843 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8acc116000 con 0x7f8acc12d590 2024-09-13T17:43:40.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.762+0000 7f8ac8ff9640 1 -- 172.21.15.102:0/377417843 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f8ac401b400 con 0x7f8acc12d590 2024-09-13T17:43:40.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.762+0000 7f8ac8ff9640 1 -- 172.21.15.102:0/377417843 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f8ac4041ab0 con 0x7f8acc12d590 2024-09-13T17:43:40.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.762+0000 7f8ad311d640 1 -- 172.21.15.102:0/377417843 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8acc117d70 con 0x7f8acc12d590 2024-09-13T17:43:40.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.766+0000 7f8ac8ff9640 1 -- 172.21.15.102:0/377417843 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f8acc117d70 con 0x7f8acc12d590 2024-09-13T17:43:40.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.948+0000 7f8ad311d640 1 -- 172.21.15.102:0/377417843 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8acc117d70 con 0x7f8acc12d590 2024-09-13T17:43:40.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.949+0000 7f8ac8ff9640 1 -- 172.21.15.102:0/377417843 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v5) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8acc117d70 con 0x7f8acc12d590 2024-09-13T17:43:40.951 INFO:teuthology.orchestra.run.smithi102.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T17:43:40.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.951+0000 7f8ad311d640 1 -- 172.21.15.102:0/377417843 >> v1:172.21.15.102:6816/34937 conn(0x7f8aac031190 legacy=0x7f8aac033640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.951+0000 7f8ad311d640 1 -- 172.21.15.102:0/377417843 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8acc12d590 msgr2=0x7f8acc118410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.951+0000 7f8ad311d640 1 --2- 172.21.15.102:0/377417843 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8acc12d590 0x7f8acc118410 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8ac400c5e0 tx=0x7f8ac4007d00 comp rx=0 tx=0).stop 2024-09-13T17:43:40.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.951+0000 7f8ad311d640 1 -- 172.21.15.102:0/377417843 shutdown_connections 2024-09-13T17:43:40.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.951+0000 7f8ad311d640 1 --2- 172.21.15.102:0/377417843 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8acc132ec0 0x7f8acc13aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.951+0000 7f8ad311d640 1 --2- 172.21.15.102:0/377417843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8acc1305a0 0x7f8acc13a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.951+0000 7f8ad311d640 1 --2- 172.21.15.102:0/377417843 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8acc12d590 0x7f8acc118410 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.951+0000 7f8ad311d640 1 -- 172.21.15.102:0/377417843 >> 172.21.15.102:0/377417843 conn(0x7f8acc05db50 msgr2=0x7f8acc09e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:40.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.951+0000 7f8ad311d640 1 -- 172.21.15.102:0/377417843 shutdown_connections 2024-09-13T17:43:40.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:40.951+0000 7f8ad311d640 1 -- 172.21.15.102:0/377417843 wait complete. 2024-09-13T17:43:40.963 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T17:43:40.964 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T17:43:40.964 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi171.front.sepia.ceph.com 2024-09-13T17:43:40.964 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-13T17:43:40.964 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T17:43:40.964 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T17:43:40.965 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T17:43:40.965 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T17:43:40.965 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T17:43:40.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:40.966 DEBUG:teuthology.orchestra.run.smithi171:> ip addr 2024-09-13T17:43:40.985 INFO:teuthology.orchestra.run.smithi171.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T17:43:40.985 INFO:teuthology.orchestra.run.smithi171.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T17:43:40.985 INFO:teuthology.orchestra.run.smithi171.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T17:43:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: inet6 ::1/128 scope host 2024-09-13T17:43:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:40.986 INFO:teuthology.orchestra.run.smithi171.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T17:43:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: link/ether 0c:c4:7a:d9:39:a2 brd ff:ff:ff:ff:ff:ff 2024-09-13T17:43:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: altname enp7s0f0 2024-09-13T17:43:40.986 INFO:teuthology.orchestra.run.smithi171.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T17:43:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: link/ether 0c:c4:7a:d9:39:a3 brd ff:ff:ff:ff:ff:ff 2024-09-13T17:43:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: altname enp7s0f1 2024-09-13T17:43:40.986 INFO:teuthology.orchestra.run.smithi171.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T17:43:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: link/ether 0c:c4:7a:88:6f:6c brd ff:ff:ff:ff:ff:ff 2024-09-13T17:43:40.986 INFO:teuthology.orchestra.run.smithi171.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T17:43:40.986 INFO:teuthology.orchestra.run.smithi171.stdout: link/ether 0c:c4:7a:88:6f:6d brd ff:ff:ff:ff:ff:ff 2024-09-13T17:43:40.987 INFO:teuthology.orchestra.run.smithi171.stdout: inet 172.21.15.171/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-13T17:43:40.987 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft 42697sec preferred_lft 42697sec 2024-09-13T17:43:40.987 INFO:teuthology.orchestra.run.smithi171.stdout: inet6 fe80::ec4:7aff:fe88:6f6d/64 scope link noprefixroute 2024-09-13T17:43:40.987 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:40.987 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-13T17:43:40.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:40.987 DEBUG:teuthology.orchestra.run.smithi171:> (cd / && exec bash -c ' 2024-09-13T17:43:40.988 DEBUG:teuthology.orchestra.run.smithi171:> set -e 2024-09-13T17:43:40.988 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip link add name ceph-brx type bridge 2024-09-13T17:43:40.988 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip addr flush dev ceph-brx 2024-09-13T17:43:40.988 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip link set ceph-brx up 2024-09-13T17:43:40.988 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-13T17:43:40.988 DEBUG:teuthology.orchestra.run.smithi171:> ') 2024-09-13T17:43:41.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:41.094 DEBUG:teuthology.orchestra.run.smithi171:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T17:43:41.120 INFO:teuthology.orchestra.run.smithi171.stdout:1 2024-09-13T17:43:41.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:41.122 DEBUG:teuthology.orchestra.run.smithi171:> route 2024-09-13T17:43:41.178 INFO:teuthology.orchestra.run.smithi171.stdout:Kernel IP routing table 2024-09-13T17:43:41.178 INFO:teuthology.orchestra.run.smithi171.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T17:43:41.178 INFO:teuthology.orchestra.run.smithi171.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-13T17:43:41.178 INFO:teuthology.orchestra.run.smithi171.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-13T17:43:41.178 INFO:teuthology.orchestra.run.smithi171.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T17:43:41.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:41.179 DEBUG:teuthology.orchestra.run.smithi171:> (cd / && exec bash -c ' 2024-09-13T17:43:41.179 DEBUG:teuthology.orchestra.run.smithi171:> set -e 2024-09-13T17:43:41.179 DEBUG:teuthology.orchestra.run.smithi171:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-13T17:43:41.179 DEBUG:teuthology.orchestra.run.smithi171:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-13T17:43:41.179 DEBUG:teuthology.orchestra.run.smithi171:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-13T17:43:41.179 DEBUG:teuthology.orchestra.run.smithi171:> ') 2024-09-13T17:43:41.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:41.268 DEBUG:teuthology.orchestra.run.smithi171:> ip netns list 2024-09-13T17:43:41.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:41.283 DEBUG:teuthology.orchestra.run.smithi171:> ip netns list-id 2024-09-13T17:43:41.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:41.338 DEBUG:teuthology.orchestra.run.smithi171:> (cd / && exec bash -c ' 2024-09-13T17:43:41.338 DEBUG:teuthology.orchestra.run.smithi171:> set -e 2024-09-13T17:43:41.338 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T17:43:41.338 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-13T17:43:41.339 DEBUG:teuthology.orchestra.run.smithi171:> ') 2024-09-13T17:43:41.416 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-13T17:43:41.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:41.417 DEBUG:teuthology.orchestra.run.smithi171:> (cd / && exec bash -c ' 2024-09-13T17:43:41.417 DEBUG:teuthology.orchestra.run.smithi171:> set -e 2024-09-13T17:43:41.417 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-13T17:43:41.417 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-13T17:43:41.417 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-13T17:43:41.417 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-13T17:43:41.417 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-13T17:43:41.417 DEBUG:teuthology.orchestra.run.smithi171:> ') 2024-09-13T17:43:41.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:41.534 DEBUG:teuthology.orchestra.run.smithi171:> (cd / && exec bash -c ' 2024-09-13T17:43:41.534 DEBUG:teuthology.orchestra.run.smithi171:> set -e 2024-09-13T17:43:41.534 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip link set brx.0 up 2024-09-13T17:43:41.534 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip link set dev brx.0 master ceph-brx 2024-09-13T17:43:41.534 DEBUG:teuthology.orchestra.run.smithi171:> ') 2024-09-13T17:43:41.571 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-13T17:43:41.572 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:43:41.572 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-13T17:43:41.627 INFO:teuthology.orchestra.run.smithi171.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-13T17:43:41.628 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:43:41.628 DEBUG:teuthology.orchestra.run.smithi171:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-13T17:43:41.683 DEBUG:teuthology.orchestra.run.smithi171:> sudo modprobe fuse 2024-09-13T17:43:41.749 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:43:41.750 DEBUG:teuthology.orchestra.run.smithi171:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:43:41.813 INFO:teuthology.orchestra.run.smithi171.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:43:41.814 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:43:41.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:41.815 DEBUG:teuthology.orchestra.run.smithi171:> ls /sys/fs/fuse/connections 2024-09-13T17:43:41.871 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-13T17:43:41.871 DEBUG:teuthology.orchestra.run.smithi171:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-13T17:43:41.914 DEBUG:teuthology.orchestra.run.smithi171:> sudo modprobe fuse 2024-09-13T17:43:41.938 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:43:41.939 DEBUG:teuthology.orchestra.run.smithi171:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:43:41.996 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi171.stderr:2024-09-13T17:43:41.994+0000 7f6808217480 -1 init, newargv = 0x562883504410 newargc=15 2024-09-13T17:43:41.997 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi171.stderr:ceph-fuse[30541]: starting ceph client 2024-09-13T17:43:42.002 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi171.stderr:ceph-fuse[30541]: starting fuse 2024-09-13T17:43:42.004 INFO:teuthology.orchestra.run.smithi171.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:43:42.006 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:43:42.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:42.006 DEBUG:teuthology.orchestra.run.smithi171:> ls /sys/fs/fuse/connections 2024-09-13T17:43:42.062 INFO:teuthology.orchestra.run.smithi171.stdout:43 2024-09-13T17:43:42.062 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-09-13T17:43:42.062 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:43:42.062 DEBUG:teuthology.orchestra.run.smithi171:> import glob 2024-09-13T17:43:42.062 DEBUG:teuthology.orchestra.run.smithi171:> import re 2024-09-13T17:43:42.062 DEBUG:teuthology.orchestra.run.smithi171:> import os 2024-09-13T17:43:42.062 DEBUG:teuthology.orchestra.run.smithi171:> import subprocess 2024-09-13T17:43:42.062 DEBUG:teuthology.orchestra.run.smithi171:> 2024-09-13T17:43:42.063 DEBUG:teuthology.orchestra.run.smithi171:> def _find_admin_socket(client_name): 2024-09-13T17:43:42.063 DEBUG:teuthology.orchestra.run.smithi171:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-13T17:43:42.063 DEBUG:teuthology.orchestra.run.smithi171:> files = glob.glob(asok_path) 2024-09-13T17:43:42.063 DEBUG:teuthology.orchestra.run.smithi171:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-13T17:43:42.063 DEBUG:teuthology.orchestra.run.smithi171:> 2024-09-13T17:43:42.063 DEBUG:teuthology.orchestra.run.smithi171:> # Given a non-glob path, it better be there 2024-09-13T17:43:42.063 DEBUG:teuthology.orchestra.run.smithi171:> if "*" not in asok_path: 2024-09-13T17:43:42.063 DEBUG:teuthology.orchestra.run.smithi171:> assert(len(files) == 1) 2024-09-13T17:43:42.063 DEBUG:teuthology.orchestra.run.smithi171:> return files[0] 2024-09-13T17:43:42.063 DEBUG:teuthology.orchestra.run.smithi171:> 2024-09-13T17:43:42.063 DEBUG:teuthology.orchestra.run.smithi171:> for f in files: 2024-09-13T17:43:42.063 DEBUG:teuthology.orchestra.run.smithi171:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:43:42.063 DEBUG:teuthology.orchestra.run.smithi171:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:43:42.063 DEBUG:teuthology.orchestra.run.smithi171:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:43:42.063 DEBUG:teuthology.orchestra.run.smithi171:> contents = proc_f.read() 2024-09-13T17:43:42.063 DEBUG:teuthology.orchestra.run.smithi171:> if mountpoint in contents: 2024-09-13T17:43:42.064 DEBUG:teuthology.orchestra.run.smithi171:> return f 2024-09-13T17:43:42.064 DEBUG:teuthology.orchestra.run.smithi171:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:43:42.064 DEBUG:teuthology.orchestra.run.smithi171:> 2024-09-13T17:43:42.064 DEBUG:teuthology.orchestra.run.smithi171:> print(_find_admin_socket("client.0")) 2024-09-13T17:43:42.064 DEBUG:teuthology.orchestra.run.smithi171:> ' 2024-09-13T17:43:42.176 INFO:teuthology.orchestra.run.smithi171.stdout:/var/run/ceph/ceph-client.0.30541.asok 2024-09-13T17:43:42.362 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.30541.asok 2024-09-13T17:43:42.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:42.362 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.30541.asok status 2024-09-13T17:43:42.496 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-13T17:43:42.496 INFO:teuthology.orchestra.run.smithi171.stdout: "metadata": { 2024-09-13T17:43:42.496 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_sha1": "6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3", 2024-09-13T17:43:42.496 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T17:43:42.496 INFO:teuthology.orchestra.run.smithi171.stdout: "entity_id": "0", 2024-09-13T17:43:42.497 INFO:teuthology.orchestra.run.smithi171.stdout: "hostname": "smithi171", 2024-09-13T17:43:42.497 INFO:teuthology.orchestra.run.smithi171.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-13T17:43:42.497 INFO:teuthology.orchestra.run.smithi171.stdout: "pid": "30541", 2024-09-13T17:43:42.497 INFO:teuthology.orchestra.run.smithi171.stdout: "root": "/" 2024-09-13T17:43:42.497 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-13T17:43:42.497 INFO:teuthology.orchestra.run.smithi171.stdout: "dentry_count": 0, 2024-09-13T17:43:42.497 INFO:teuthology.orchestra.run.smithi171.stdout: "dentry_pinned_count": 0, 2024-09-13T17:43:42.498 INFO:teuthology.orchestra.run.smithi171.stdout: "id": 4790, 2024-09-13T17:43:42.498 INFO:teuthology.orchestra.run.smithi171.stdout: "inst": { 2024-09-13T17:43:42.498 INFO:teuthology.orchestra.run.smithi171.stdout: "name": { 2024-09-13T17:43:42.498 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "client", 2024-09-13T17:43:42.498 INFO:teuthology.orchestra.run.smithi171.stdout: "num": 4790 2024-09-13T17:43:42.498 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-13T17:43:42.498 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": { 2024-09-13T17:43:42.498 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-13T17:43:42.499 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:0", 2024-09-13T17:43:42.499 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 761743079 2024-09-13T17:43:42.499 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-13T17:43:42.499 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-13T17:43:42.499 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": { 2024-09-13T17:43:42.499 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-13T17:43:42.499 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:0", 2024-09-13T17:43:42.500 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 761743079 2024-09-13T17:43:42.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-13T17:43:42.500 INFO:teuthology.orchestra.run.smithi171.stdout: "inst_str": "client.4790 172.21.15.171:0/761743079", 2024-09-13T17:43:42.500 INFO:teuthology.orchestra.run.smithi171.stdout: "addr_str": "172.21.15.171:0/761743079", 2024-09-13T17:43:42.500 INFO:teuthology.orchestra.run.smithi171.stdout: "inode_count": 1, 2024-09-13T17:43:42.500 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_epoch": 5, 2024-09-13T17:43:42.501 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_epoch": 23, 2024-09-13T17:43:42.501 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_epoch_barrier": 0, 2024-09-13T17:43:42.501 INFO:teuthology.orchestra.run.smithi171.stdout: "blocklisted": false, 2024-09-13T17:43:42.501 INFO:teuthology.orchestra.run.smithi171.stdout: "fs_name": "cephfs" 2024-09-13T17:43:42.501 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-13T17:43:42.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:42.505 DEBUG:teuthology.orchestra.run.smithi171:> stat --file-system '--printf=%T 2024-09-13T17:43:42.505 DEBUG:teuthology.orchestra.run.smithi171:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-13T17:43:42.521 INFO:teuthology.orchestra.run.smithi171.stdout:fuseblk 2024-09-13T17:43:42.522 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-13T17:43:42.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:42.522 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-13T17:43:42.586 INFO:teuthology.run_tasks:Running task print... 2024-09-13T17:43:42.595 INFO:teuthology.task.print:**** done remount client.0 2024-09-13T17:43:42.595 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-13T17:43:42.611 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:43:42.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.721+0000 7f135e914640 1 Processor -- start 2024-09-13T17:43:42.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.721+0000 7f135e914640 1 -- start start 2024-09-13T17:43:42.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.721+0000 7f135e914640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1358068920 0x7f135809d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:42.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.721+0000 7f135e914640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f135809dc50 0x7f135809e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:42.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.721+0000 7f135e914640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f135812f310 0x7f135812f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:42.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.721+0000 7f135e914640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1358068cf0 con 0x7f135812f310 2024-09-13T17:43:42.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.721+0000 7f135e914640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13580695a0 con 0x7f135809dc50 2024-09-13T17:43:42.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.721+0000 7f135e914640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f135809f090 con 0x7f1358068920 2024-09-13T17:43:42.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.721+0000 7f13577fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f135809dc50 0x7f135809e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:42.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.721+0000 7f1357fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1358068920 0x7f135809d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:42.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.721+0000 7f135ceb1640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f135812f310 0x7f135812f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:42.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.721+0000 7f13577fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f135809dc50 0x7f135809e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:50688/0 (socket says 172.21.15.102:50688) 2024-09-13T17:43:42.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.721+0000 7f13577fe640 1 -- 172.21.15.102:0/1306823728 learned_addr learned my addr 172.21.15.102:0/1306823728 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:42.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.722+0000 7f1357fff640 1 -- 172.21.15.102:0/1306823728 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f135809dc50 msgr2=0x7f135809e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:42.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.722+0000 7f1357fff640 1 --2- 172.21.15.102:0/1306823728 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f135809dc50 0x7f135809e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:42.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.722+0000 7f1357fff640 1 -- 172.21.15.102:0/1306823728 >> v2:172.21.15.102:3300/0 conn(0x7f135812f310 msgr2=0x7f135812f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:42.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.722+0000 7f1357fff640 1 --2- 172.21.15.102:0/1306823728 >> v2:172.21.15.102:3300/0 conn(0x7f135812f310 0x7f135812f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:42.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.722+0000 7f1357fff640 1 -- 172.21.15.102:0/1306823728 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f135809cce0 con 0x7f1358068920 2024-09-13T17:43:42.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.722+0000 7f135ceb1640 1 --2- 172.21.15.102:0/1306823728 >> v2:172.21.15.102:3300/0 conn(0x7f135812f310 0x7f135812f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:42.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.722+0000 7f13577fe640 1 --2- 172.21.15.102:0/1306823728 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f135809dc50 0x7f135809e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:42.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.722+0000 7f1357fff640 1 --2- 172.21.15.102:0/1306823728 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1358068920 0x7f135809d710 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1348009850 tx=0x7f13480090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6686c7881b11f6b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:42.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.723+0000 7f1356ffd640 1 -- 172.21.15.102:0/1306823728 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134802ac50 con 0x7f1358068920 2024-09-13T17:43:42.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.723+0000 7f1356ffd640 1 -- 172.21.15.102:0/1306823728 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f134802adb0 con 0x7f1358068920 2024-09-13T17:43:42.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.723+0000 7f1356ffd640 1 -- 172.21.15.102:0/1306823728 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1348032670 con 0x7f1358068920 2024-09-13T17:43:42.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.723+0000 7f135e914640 1 -- 172.21.15.102:0/1306823728 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1358068920 msgr2=0x7f135809d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:42.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.723+0000 7f135e914640 1 --2- 172.21.15.102:0/1306823728 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1358068920 0x7f135809d710 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1348009850 tx=0x7f13480090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:42.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.723+0000 7f135e914640 1 -- 172.21.15.102:0/1306823728 shutdown_connections 2024-09-13T17:43:42.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.723+0000 7f135e914640 1 --2- 172.21.15.102:0/1306823728 >> v2:172.21.15.102:3300/0 conn(0x7f135812f310 0x7f135812f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:42.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.723+0000 7f135e914640 1 --2- 172.21.15.102:0/1306823728 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f135809dc50 0x7f135809e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:42.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.723+0000 7f135e914640 1 --2- 172.21.15.102:0/1306823728 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1358068920 0x7f135809d710 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:42.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.723+0000 7f135e914640 1 -- 172.21.15.102:0/1306823728 >> 172.21.15.102:0/1306823728 conn(0x7f135806f320 msgr2=0x7f13580a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:42.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.723+0000 7f135e914640 1 -- 172.21.15.102:0/1306823728 shutdown_connections 2024-09-13T17:43:42.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.723+0000 7f135e914640 1 -- 172.21.15.102:0/1306823728 wait complete. 2024-09-13T17:43:42.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.724+0000 7f135e914640 1 Processor -- start 2024-09-13T17:43:42.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.725+0000 7f135e914640 1 -- start start 2024-09-13T17:43:42.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.725+0000 7f135e914640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1358068920 0x7f135811de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:42.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.725+0000 7f135e914640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f135809dc50 0x7f135811e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:42.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.725+0000 7f135e914640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f135812f310 0x7f135811e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:42.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.725+0000 7f135e914640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135809fee0 con 0x7f1358068920 2024-09-13T17:43:42.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.725+0000 7f135e914640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1358121e60 con 0x7f135809dc50 2024-09-13T17:43:42.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.725+0000 7f135e914640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f135811f240 con 0x7f135812f310 2024-09-13T17:43:42.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.725+0000 7f1357fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1358068920 0x7f135811de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:42.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.725+0000 7f135ceb1640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f135812f310 0x7f135811e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:42.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.725+0000 7f1357fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1358068920 0x7f135811de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57454/0 (socket says 172.21.15.102:57454) 2024-09-13T17:43:42.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.725+0000 7f1357fff640 1 -- 172.21.15.102:0/2678140966 learned_addr learned my addr 172.21.15.102:0/2678140966 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:42.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.725+0000 7f13577fe640 1 --2- 172.21.15.102:0/2678140966 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f135809dc50 0x7f135811e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:42.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.725+0000 7f135ceb1640 1 -- 172.21.15.102:0/2678140966 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f135809dc50 msgr2=0x7f135811e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:42.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.725+0000 7f135ceb1640 1 --2- 172.21.15.102:0/2678140966 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f135809dc50 0x7f135811e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:42.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.725+0000 7f135ceb1640 1 -- 172.21.15.102:0/2678140966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1358068920 msgr2=0x7f135811de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:42.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.725+0000 7f135ceb1640 1 --2- 172.21.15.102:0/2678140966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1358068920 0x7f135811de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:42.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.725+0000 7f135ceb1640 1 -- 172.21.15.102:0/2678140966 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f134c009450 con 0x7f135812f310 2024-09-13T17:43:42.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.725+0000 7f13577fe640 1 --2- 172.21.15.102:0/2678140966 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f135809dc50 0x7f135811e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:42.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.725+0000 7f1357fff640 1 --2- 172.21.15.102:0/2678140966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1358068920 0x7f135811de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:42.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.726+0000 7f135ceb1640 1 --2- 172.21.15.102:0/2678140966 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f135812f310 0x7f135811e8e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1350009f10 tx=0x7f135000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:42.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.726+0000 7f133ffff640 1 -- 172.21.15.102:0/2678140966 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135000cdb0 con 0x7f135812f310 2024-09-13T17:43:42.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.726+0000 7f133ffff640 1 -- 172.21.15.102:0/2678140966 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f135000f040 con 0x7f135812f310 2024-09-13T17:43:42.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.726+0000 7f135e914640 1 -- 172.21.15.102:0/2678140966 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1350009a20 con 0x7f135812f310 2024-09-13T17:43:42.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.726+0000 7f133ffff640 1 -- 172.21.15.102:0/2678140966 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13500146d0 con 0x7f135812f310 2024-09-13T17:43:42.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.726+0000 7f135e914640 1 -- 172.21.15.102:0/2678140966 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1348009390 con 0x7f135812f310 2024-09-13T17:43:42.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.727+0000 7f133ffff640 1 -- 172.21.15.102:0/2678140966 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f1350003020 con 0x7f135812f310 2024-09-13T17:43:42.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.727+0000 7f133ffff640 1 -- 172.21.15.102:0/2678140966 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f1350042b60 con 0x7f135812f310 2024-09-13T17:43:42.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.727+0000 7f135e914640 1 -- 172.21.15.102:0/2678140966 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1358064000 con 0x7f135812f310 2024-09-13T17:43:42.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.732+0000 7f133ffff640 1 -- 172.21.15.102:0/2678140966 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f1350018070 con 0x7f135812f310 2024-09-13T17:43:42.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.884+0000 7f135e914640 1 -- 172.21.15.102:0/2678140966 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f135809c7d0 con 0x7f135812f310 2024-09-13T17:43:42.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.885+0000 7f133ffff640 1 -- 172.21.15.102:0/2678140966 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+11660 (secure 0 0 0) 0x7f13500133e0 con 0x7f135812f310 2024-09-13T17:43:42.888 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:43:42.888 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":23,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:42:48.385896+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:04.672265+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:05.583134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.17","mappings":[{"from":3,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]},{"pgid":"4.25","mappings":[{"from":3,"to":2}]},{"pgid":"4.33","mappings":[{"from":3,"to":0}]},{"pgid":"4.3a","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:43:42.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.887+0000 7f135e914640 1 -- 172.21.15.102:0/2678140966 >> v1:172.21.15.102:6816/34937 conn(0x7f1344031160 legacy=0x7f1344033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:42.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.887+0000 7f135e914640 1 -- 172.21.15.102:0/2678140966 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f135812f310 msgr2=0x7f135811e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:42.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.887+0000 7f135e914640 1 --2- 172.21.15.102:0/2678140966 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f135812f310 0x7f135811e8e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1350009f10 tx=0x7f135000b040 comp rx=0 tx=0).stop 2024-09-13T17:43:42.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.887+0000 7f135e914640 1 -- 172.21.15.102:0/2678140966 shutdown_connections 2024-09-13T17:43:42.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.887+0000 7f135e914640 1 --2- 172.21.15.102:0/2678140966 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f135812f310 0x7f135811e8e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:42.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.887+0000 7f135e914640 1 --2- 172.21.15.102:0/2678140966 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f135809dc50 0x7f135811e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:42.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.887+0000 7f135e914640 1 --2- 172.21.15.102:0/2678140966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1358068920 0x7f135811de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:42.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.887+0000 7f135e914640 1 -- 172.21.15.102:0/2678140966 >> 172.21.15.102:0/2678140966 conn(0x7f135806f320 msgr2=0x7f135809afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:42.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.887+0000 7f135e914640 1 -- 172.21.15.102:0/2678140966 shutdown_connections 2024-09-13T17:43:42.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.887+0000 7f135e914640 1 -- 172.21.15.102:0/2678140966 wait complete. 2024-09-13T17:43:42.900 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:54.442612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T17:41:59.076782+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 16, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:42:04.672265+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:42:05.583134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:43:42.901 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:43:43.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.995+0000 7fd8754a8640 1 Processor -- start 2024-09-13T17:43:43.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.995+0000 7fd8754a8640 1 -- start start 2024-09-13T17:43:43.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.995+0000 7fd8754a8640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd8700a10f0 0x7fd87012f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:43.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.995+0000 7fd8754a8640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd87012fff0 0x7fd87009eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:43.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.995+0000 7fd8754a8640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd870069000 con 0x7fd87012fbf0 2024-09-13T17:43:43.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.995+0000 7fd8754a8640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8700a14c0 con 0x7fd8700a10f0 2024-09-13T17:43:43.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.995+0000 7fd8754a8640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8700a0100 con 0x7fd87012fff0 2024-09-13T17:43:43.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.996+0000 7fd86f7fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd87012fff0 0x7fd87009eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:43.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.996+0000 7fd86effd640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd8700a10f0 0x7fd87012f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:43.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.996+0000 7fd86e7fc640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fd87012fbf0 0x7fd87009e8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36290/0 (socket says 172.21.15.102:36290) 2024-09-13T17:43:43.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.996+0000 7fd86e7fc640 1 -- 172.21.15.102:0/2576325484 learned_addr learned my addr 172.21.15.102:0/2576325484 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:43.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.996+0000 7fd86f7fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd87012fff0 0x7fd87009eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:50704/0 (socket says 172.21.15.102:50704) 2024-09-13T17:43:43.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.996+0000 7fd86f7fe640 1 -- 172.21.15.102:0/2576325484 >> v2:172.21.15.102:3300/0 conn(0x7fd8700a10f0 msgr2=0x7fd87012f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:43.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.996+0000 7fd86f7fe640 1 --2- 172.21.15.102:0/2576325484 >> v2:172.21.15.102:3300/0 conn(0x7fd8700a10f0 0x7fd87012f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.996+0000 7fd86f7fe640 1 -- 172.21.15.102:0/2576325484 >> v1:172.21.15.102:6789/0 conn(0x7fd87012fbf0 legacy=0x7fd87009e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:43.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.996+0000 7fd86dffb640 1 -- 172.21.15.102:0/2576325484 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1445798241 0 0) 0x7fd870069000 con 0x7fd87012fbf0 2024-09-13T17:43:43.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.996+0000 7fd86f7fe640 1 -- 172.21.15.102:0/2576325484 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd870068240 con 0x7fd87012fff0 2024-09-13T17:43:43.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.996+0000 7fd86effd640 1 --2- 172.21.15.102:0/2576325484 >> v2:172.21.15.102:3300/0 conn(0x7fd8700a10f0 0x7fd87012f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:43.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.997+0000 7fd86f7fe640 1 --2- 172.21.15.102:0/2576325484 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd87012fff0 0x7fd87009eff0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd868009e60 tx=0x7fd868009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4826534cf7b5cbdd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:43.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.997+0000 7fd86dffb640 1 -- 172.21.15.102:0/2576325484 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd868007550 con 0x7fd87012fff0 2024-09-13T17:43:43.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.997+0000 7fd86dffb640 1 -- 172.21.15.102:0/2576325484 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd86802d050 con 0x7fd87012fff0 2024-09-13T17:43:43.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.997+0000 7fd86dffb640 1 -- 172.21.15.102:0/2576325484 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8680078e0 con 0x7fd87012fff0 2024-09-13T17:43:43.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.997+0000 7fd8754a8640 1 -- 172.21.15.102:0/2576325484 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd87012fff0 msgr2=0x7fd87009eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:43.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.997+0000 7fd8754a8640 1 --2- 172.21.15.102:0/2576325484 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd87012fff0 0x7fd87009eff0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd868009e60 tx=0x7fd868009780 comp rx=0 tx=0).stop 2024-09-13T17:43:43.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.997+0000 7fd8754a8640 1 -- 172.21.15.102:0/2576325484 shutdown_connections 2024-09-13T17:43:43.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.997+0000 7fd8754a8640 1 --2- 172.21.15.102:0/2576325484 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd87012fff0 0x7fd87009eff0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.997+0000 7fd8754a8640 1 --2- 172.21.15.102:0/2576325484 >> v2:172.21.15.102:3300/0 conn(0x7fd8700a10f0 0x7fd87012f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.997+0000 7fd8754a8640 1 -- 172.21.15.102:0/2576325484 >> 172.21.15.102:0/2576325484 conn(0x7fd870067670 msgr2=0x7fd8700a09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:43.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.997+0000 7fd8754a8640 1 -- 172.21.15.102:0/2576325484 shutdown_connections 2024-09-13T17:43:43.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.997+0000 7fd8754a8640 1 -- 172.21.15.102:0/2576325484 wait complete. 2024-09-13T17:43:43.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.998+0000 7fd8754a8640 1 Processor -- start 2024-09-13T17:43:43.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.998+0000 7fd8754a8640 1 -- start start 2024-09-13T17:43:43.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.999+0000 7fd8754a8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8700a10f0 0x7fd87011c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:43.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.999+0000 7fd8754a8640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd87012fbf0 0x7fd870119020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:43.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.999+0000 7fd8754a8640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd87012fff0 0x7fd870119560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:43.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.999+0000 7fd8754a8640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd870068680 con 0x7fd8700a10f0 2024-09-13T17:43:43.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.999+0000 7fd8754a8640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd87011bb90 con 0x7fd87012fff0 2024-09-13T17:43:43.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.999+0000 7fd8754a8640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd87009b7a0 con 0x7fd87012fbf0 2024-09-13T17:43:43.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.999+0000 7fd86effd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8700a10f0 0x7fd87011c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:43.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.999+0000 7fd86e7fc640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd87012fbf0 0x7fd870119020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:43.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.999+0000 7fd86effd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8700a10f0 0x7fd87011c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57484/0 (socket says 172.21.15.102:57484) 2024-09-13T17:43:43.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.999+0000 7fd86effd640 1 -- 172.21.15.102:0/4133498361 learned_addr learned my addr 172.21.15.102:0/4133498361 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:43.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.999+0000 7fd86f7fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd87012fff0 0x7fd870119560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:43.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.999+0000 7fd86effd640 1 -- 172.21.15.102:0/4133498361 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd87012fbf0 msgr2=0x7fd870119020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:43.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.999+0000 7fd86effd640 1 --2- 172.21.15.102:0/4133498361 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd87012fbf0 0x7fd870119020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.999+0000 7fd86effd640 1 -- 172.21.15.102:0/4133498361 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd87012fff0 msgr2=0x7fd870119560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:43.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.999+0000 7fd86effd640 1 --2- 172.21.15.102:0/4133498361 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd87012fff0 0x7fd870119560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.999+0000 7fd86effd640 1 -- 172.21.15.102:0/4133498361 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd860009390 con 0x7fd8700a10f0 2024-09-13T17:43:43.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.999+0000 7fd86f7fe640 1 --2- 172.21.15.102:0/4133498361 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd87012fff0 0x7fd870119560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:43.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.999+0000 7fd86e7fc640 1 --2- 172.21.15.102:0/4133498361 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd87012fbf0 0x7fd870119020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:43.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.999+0000 7fd86effd640 1 --2- 172.21.15.102:0/4133498361 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8700a10f0 0x7fd87011c630 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd86000ec80 tx=0x7fd8600090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:43.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.999+0000 7fd856ffd640 1 -- 172.21.15.102:0/4133498361 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86000eec0 con 0x7fd8700a10f0 2024-09-13T17:43:43.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:42.999+0000 7fd856ffd640 1 -- 172.21.15.102:0/4133498361 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd86000cbc0 con 0x7fd8700a10f0 2024-09-13T17:43:43.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.000+0000 7fd856ffd640 1 -- 172.21.15.102:0/4133498361 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd860013470 con 0x7fd8700a10f0 2024-09-13T17:43:43.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.000+0000 7fd8754a8640 1 -- 172.21.15.102:0/4133498361 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd868009a20 con 0x7fd8700a10f0 2024-09-13T17:43:43.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.000+0000 7fd8754a8640 1 -- 172.21.15.102:0/4133498361 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd870118d60 con 0x7fd8700a10f0 2024-09-13T17:43:43.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.000+0000 7fd856ffd640 1 -- 172.21.15.102:0/4133498361 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fd8600137e0 con 0x7fd8700a10f0 2024-09-13T17:43:43.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.001+0000 7fd8754a8640 1 -- 172.21.15.102:0/4133498361 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd87011bdb0 con 0x7fd8700a10f0 2024-09-13T17:43:43.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.001+0000 7fd856ffd640 1 -- 172.21.15.102:0/4133498361 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7fd860042830 con 0x7fd8700a10f0 2024-09-13T17:43:43.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.005+0000 7fd856ffd640 1 -- 172.21.15.102:0/4133498361 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fd86001bb90 con 0x7fd8700a10f0 2024-09-13T17:43:43.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.156+0000 7fd8754a8640 1 -- 172.21.15.102:0/4133498361 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd870064000 con 0x7fd8700a10f0 2024-09-13T17:43:43.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.156+0000 7fd856ffd640 1 -- 172.21.15.102:0/4133498361 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd860021070 con 0x7fd8700a10f0 2024-09-13T17:43:43.159 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-09-13T17:43:43.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.158+0000 7fd8754a8640 1 -- 172.21.15.102:0/4133498361 >> v1:172.21.15.102:6816/34937 conn(0x7fd85c031160 legacy=0x7fd85c033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:43.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.158+0000 7fd8754a8640 1 -- 172.21.15.102:0/4133498361 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8700a10f0 msgr2=0x7fd87011c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:43.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.158+0000 7fd8754a8640 1 --2- 172.21.15.102:0/4133498361 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8700a10f0 0x7fd87011c630 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd86000ec80 tx=0x7fd8600090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.158+0000 7fd8754a8640 1 -- 172.21.15.102:0/4133498361 shutdown_connections 2024-09-13T17:43:43.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.158+0000 7fd8754a8640 1 --2- 172.21.15.102:0/4133498361 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd87012fff0 0x7fd870119560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.158+0000 7fd8754a8640 1 --2- 172.21.15.102:0/4133498361 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd87012fbf0 0x7fd870119020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.158+0000 7fd8754a8640 1 --2- 172.21.15.102:0/4133498361 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8700a10f0 0x7fd87011c630 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.158+0000 7fd8754a8640 1 -- 172.21.15.102:0/4133498361 >> 172.21.15.102:0/4133498361 conn(0x7fd870067670 msgr2=0x7fd87012e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:43.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.158+0000 7fd8754a8640 1 -- 172.21.15.102:0/4133498361 shutdown_connections 2024-09-13T17:43:43.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.159+0000 7fd8754a8640 1 -- 172.21.15.102:0/4133498361 wait complete. 2024-09-13T17:43:43.170 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-13T17:43:43.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.265+0000 7fd70f480640 1 Processor -- start 2024-09-13T17:43:43.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.265+0000 7fd70f480640 1 -- start start 2024-09-13T17:43:43.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.265+0000 7fd70f480640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd708096a70 0x7fd7080987c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:43.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.265+0000 7fd70f480640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd708099110 0x7fd70812c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:43.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.266+0000 7fd70f480640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7080999e0 con 0x7fd708098d00 2024-09-13T17:43:43.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.266+0000 7fd70f480640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd708069c30 con 0x7fd708099110 2024-09-13T17:43:43.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.266+0000 7fd70f480640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd70806ed00 con 0x7fd708096a70 2024-09-13T17:43:43.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.266+0000 7fd70d21c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd708096a70 0x7fd7080987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:43.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.266+0000 7fd70ca1b640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fd708098d00 0x7fd70812c2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36298/0 (socket says 172.21.15.102:36298) 2024-09-13T17:43:43.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.266+0000 7fd70da1d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd708099110 0x7fd70812c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:43.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.266+0000 7fd70ca1b640 1 -- 172.21.15.102:0/2110156571 learned_addr learned my addr 172.21.15.102:0/2110156571 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:43.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.266+0000 7fd70da1d640 1 -- 172.21.15.102:0/2110156571 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd708096a70 msgr2=0x7fd7080987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:43.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.266+0000 7fd70da1d640 1 --2- 172.21.15.102:0/2110156571 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd708096a70 0x7fd7080987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.266+0000 7fd70da1d640 1 -- 172.21.15.102:0/2110156571 >> v1:172.21.15.102:6789/0 conn(0x7fd708098d00 legacy=0x7fd70812c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:43.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.266+0000 7fd6fffff640 1 -- 172.21.15.102:0/2110156571 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3389169952 0 0) 0x7fd7080999e0 con 0x7fd708098d00 2024-09-13T17:43:43.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.266+0000 7fd70da1d640 1 -- 172.21.15.102:0/2110156571 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7080997a0 con 0x7fd708099110 2024-09-13T17:43:43.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.266+0000 7fd70d21c640 1 --2- 172.21.15.102:0/2110156571 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd708096a70 0x7fd7080987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:43.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.267+0000 7fd70da1d640 1 --2- 172.21.15.102:0/2110156571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd708099110 0x7fd70812c9f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd700009e60 tx=0x7fd700009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b149247fb02a3f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:43.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.267+0000 7fd6fffff640 1 -- 172.21.15.102:0/2110156571 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd700007550 con 0x7fd708099110 2024-09-13T17:43:43.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.267+0000 7fd6fffff640 1 -- 172.21.15.102:0/2110156571 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd70002d050 con 0x7fd708099110 2024-09-13T17:43:43.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.267+0000 7fd6fffff640 1 -- 172.21.15.102:0/2110156571 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7000078e0 con 0x7fd708099110 2024-09-13T17:43:43.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.267+0000 7fd70f480640 1 -- 172.21.15.102:0/2110156571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd708099110 msgr2=0x7fd70812c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:43.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.267+0000 7fd70f480640 1 --2- 172.21.15.102:0/2110156571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd708099110 0x7fd70812c9f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd700009e60 tx=0x7fd700009780 comp rx=0 tx=0).stop 2024-09-13T17:43:43.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.267+0000 7fd70f480640 1 -- 172.21.15.102:0/2110156571 shutdown_connections 2024-09-13T17:43:43.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.267+0000 7fd70f480640 1 --2- 172.21.15.102:0/2110156571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd708099110 0x7fd70812c9f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.267+0000 7fd70f480640 1 --2- 172.21.15.102:0/2110156571 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd708096a70 0x7fd7080987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.267+0000 7fd70f480640 1 -- 172.21.15.102:0/2110156571 >> 172.21.15.102:0/2110156571 conn(0x7fd70809e720 msgr2=0x7fd70809eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:43.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.267+0000 7fd70f480640 1 -- 172.21.15.102:0/2110156571 shutdown_connections 2024-09-13T17:43:43.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.267+0000 7fd70f480640 1 -- 172.21.15.102:0/2110156571 wait complete. 2024-09-13T17:43:43.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.268+0000 7fd70f480640 1 Processor -- start 2024-09-13T17:43:43.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.268+0000 7fd70f480640 1 -- start start 2024-09-13T17:43:43.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.269+0000 7fd70f480640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd708096a70 0x7fd70813a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:43.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.269+0000 7fd70f480640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd708098d00 0x7fd70813aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:43.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.269+0000 7fd70f480640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd708099110 0x7fd708117180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:43.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.269+0000 7fd70f480640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70812d310 con 0x7fd708098d00 2024-09-13T17:43:43.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.269+0000 7fd70f480640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd70809a410 con 0x7fd708096a70 2024-09-13T17:43:43.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.269+0000 7fd70f480640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd70811aba0 con 0x7fd708099110 2024-09-13T17:43:43.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.269+0000 7fd70d21c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd708096a70 0x7fd70813a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:43.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.269+0000 7fd70ca1b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd708098d00 0x7fd70813aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:43.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.269+0000 7fd70d21c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd708096a70 0x7fd70813a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:50724/0 (socket says 172.21.15.102:50724) 2024-09-13T17:43:43.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.269+0000 7fd70d21c640 1 -- 172.21.15.102:0/3967040865 learned_addr learned my addr 172.21.15.102:0/3967040865 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:43.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.269+0000 7fd70da1d640 1 --2- 172.21.15.102:0/3967040865 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd708099110 0x7fd708117180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:43.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.269+0000 7fd70d21c640 1 -- 172.21.15.102:0/3967040865 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd708099110 msgr2=0x7fd708117180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:43.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.269+0000 7fd70d21c640 1 --2- 172.21.15.102:0/3967040865 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd708099110 0x7fd708117180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.269+0000 7fd70d21c640 1 -- 172.21.15.102:0/3967040865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd708098d00 msgr2=0x7fd70813aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:43.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.269+0000 7fd70d21c640 1 --2- 172.21.15.102:0/3967040865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd708098d00 0x7fd70813aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.269+0000 7fd70d21c640 1 -- 172.21.15.102:0/3967040865 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd704009390 con 0x7fd708096a70 2024-09-13T17:43:43.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.269+0000 7fd70ca1b640 1 --2- 172.21.15.102:0/3967040865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd708098d00 0x7fd70813aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:43.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.269+0000 7fd70d21c640 1 --2- 172.21.15.102:0/3967040865 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd708096a70 0x7fd70813a570 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd70400ec80 tx=0x7fd7040090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:43.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.269+0000 7fd6fd7fa640 1 -- 172.21.15.102:0/3967040865 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70400eec0 con 0x7fd708096a70 2024-09-13T17:43:43.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.270+0000 7fd6fd7fa640 1 -- 172.21.15.102:0/3967040865 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd70400cbc0 con 0x7fd708096a70 2024-09-13T17:43:43.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.270+0000 7fd70f480640 1 -- 172.21.15.102:0/3967040865 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd700009a20 con 0x7fd708096a70 2024-09-13T17:43:43.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.270+0000 7fd6fd7fa640 1 -- 172.21.15.102:0/3967040865 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd704013470 con 0x7fd708096a70 2024-09-13T17:43:43.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.270+0000 7fd70f480640 1 -- 172.21.15.102:0/3967040865 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd708119640 con 0x7fd708096a70 2024-09-13T17:43:43.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.270+0000 7fd6fd7fa640 1 -- 172.21.15.102:0/3967040865 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fd7040137e0 con 0x7fd708096a70 2024-09-13T17:43:43.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.270+0000 7fd6fd7fa640 1 -- 172.21.15.102:0/3967040865 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7fd704042580 con 0x7fd708096a70 2024-09-13T17:43:43.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.271+0000 7fd70f480640 1 -- 172.21.15.102:0/3967040865 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd708064000 con 0x7fd708096a70 2024-09-13T17:43:43.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.275+0000 7fd6fd7fa640 1 -- 172.21.15.102:0/3967040865 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fd704023ad0 con 0x7fd708096a70 2024-09-13T17:43:43.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.430+0000 7fd70f480640 1 -- 172.21.15.102:0/3967040865 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd70811a560 con 0x7fd708096a70 2024-09-13T17:43:43.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.431+0000 7fd6fd7fa640 1 -- 172.21.15.102:0/3967040865 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd7040232f0 con 0x7fd708096a70 2024-09-13T17:43:43.433 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-09-13T17:43:43.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.433+0000 7fd70f480640 1 -- 172.21.15.102:0/3967040865 >> v1:172.21.15.102:6816/34937 conn(0x7fd6ec031160 legacy=0x7fd6ec033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:43.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.433+0000 7fd70f480640 1 -- 172.21.15.102:0/3967040865 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd708096a70 msgr2=0x7fd70813a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:43.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.433+0000 7fd70f480640 1 --2- 172.21.15.102:0/3967040865 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd708096a70 0x7fd70813a570 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd70400ec80 tx=0x7fd7040090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.433+0000 7fd70f480640 1 -- 172.21.15.102:0/3967040865 shutdown_connections 2024-09-13T17:43:43.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.433+0000 7fd70f480640 1 --2- 172.21.15.102:0/3967040865 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd708099110 0x7fd708117180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.433+0000 7fd70f480640 1 --2- 172.21.15.102:0/3967040865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd708098d00 0x7fd70813aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.433+0000 7fd70f480640 1 --2- 172.21.15.102:0/3967040865 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd708096a70 0x7fd70813a570 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.433+0000 7fd70f480640 1 -- 172.21.15.102:0/3967040865 >> 172.21.15.102:0/3967040865 conn(0x7fd70809e720 msgr2=0x7fd7080971f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:43.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.433+0000 7fd70f480640 1 -- 172.21.15.102:0/3967040865 shutdown_connections 2024-09-13T17:43:43.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.433+0000 7fd70f480640 1 -- 172.21.15.102:0/3967040865 wait complete. 2024-09-13T17:43:43.445 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:43:43.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.541+0000 7f7d6935d640 1 Processor -- start 2024-09-13T17:43:43.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.541+0000 7f7d6935d640 1 -- start start 2024-09-13T17:43:43.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.542+0000 7f7d6935d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7d64067130 0x7f7d6412d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:43.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.542+0000 7f7d6935d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f7d6412df00 0x7f7d64099390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:43.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.542+0000 7f7d6935d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d6409be00 con 0x7f7d6412db30 2024-09-13T17:43:43.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.542+0000 7f7d6935d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d64067500 con 0x7f7d6412df00 2024-09-13T17:43:43.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.542+0000 7f7d6935d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d640680f0 con 0x7f7d64067130 2024-09-13T17:43:43.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.542+0000 7f7d62ffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7d64067130 0x7f7d6412d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:43.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.542+0000 7f7d637fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f7d6412df00 0x7f7d64099390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:43.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.542+0000 7f7d62ffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7d64067130 0x7f7d6412d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:43178/0 (socket says 172.21.15.102:43178) 2024-09-13T17:43:43.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.542+0000 7f7d637fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f7d6412df00 0x7f7d64099390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57508/0 (socket says 172.21.15.102:57508) 2024-09-13T17:43:43.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.542+0000 7f7d62ffd640 1 -- 172.21.15.102:0/807611813 learned_addr learned my addr 172.21.15.102:0/807611813 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:43.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.542+0000 7f7d62ffd640 1 -- 172.21.15.102:0/807611813 >> v2:172.21.15.102:3300/0 conn(0x7f7d6412df00 msgr2=0x7f7d64099390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:43.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.542+0000 7f7d62ffd640 1 --2- 172.21.15.102:0/807611813 >> v2:172.21.15.102:3300/0 conn(0x7f7d6412df00 0x7f7d64099390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.542+0000 7f7d62ffd640 1 -- 172.21.15.102:0/807611813 >> v1:172.21.15.102:6789/0 conn(0x7f7d6412db30 legacy=0x7f7d64098c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:43.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.542+0000 7f7d62ffd640 1 -- 172.21.15.102:0/807611813 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d6409a430 con 0x7f7d64067130 2024-09-13T17:43:43.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.542+0000 7f7d627fc640 1 -- 172.21.15.102:0/807611813 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3163200430 0 0) 0x7f7d6409be00 con 0x7f7d6412db30 2024-09-13T17:43:43.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.542+0000 7f7d637fe640 1 --2- 172.21.15.102:0/807611813 >> v2:172.21.15.102:3300/0 conn(0x7f7d6412df00 0x7f7d64099390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:43.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.543+0000 7f7d62ffd640 1 --2- 172.21.15.102:0/807611813 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7d64067130 0x7f7d6412d5f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7d54009910 tx=0x7f7d540091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2978d5c2d27b4f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:43.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.543+0000 7f7d627fc640 1 -- 172.21.15.102:0/807611813 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5402adb0 con 0x7f7d64067130 2024-09-13T17:43:43.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.543+0000 7f7d627fc640 1 -- 172.21.15.102:0/807611813 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d54032440 con 0x7f7d64067130 2024-09-13T17:43:43.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.543+0000 7f7d627fc640 1 -- 172.21.15.102:0/807611813 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d540327e0 con 0x7f7d64067130 2024-09-13T17:43:43.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.544+0000 7f7d6935d640 1 -- 172.21.15.102:0/807611813 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7d64067130 msgr2=0x7f7d6412d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:43.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.544+0000 7f7d6935d640 1 --2- 172.21.15.102:0/807611813 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7d64067130 0x7f7d6412d5f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7d54009910 tx=0x7f7d540091d0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.544+0000 7f7d6935d640 1 -- 172.21.15.102:0/807611813 shutdown_connections 2024-09-13T17:43:43.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.544+0000 7f7d6935d640 1 --2- 172.21.15.102:0/807611813 >> v2:172.21.15.102:3300/0 conn(0x7f7d6412df00 0x7f7d64099390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.544+0000 7f7d6935d640 1 --2- 172.21.15.102:0/807611813 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7d64067130 0x7f7d6412d5f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.544+0000 7f7d6935d640 1 -- 172.21.15.102:0/807611813 >> 172.21.15.102:0/807611813 conn(0x7f7d6405e770 msgr2=0x7f7d6409f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:43.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.544+0000 7f7d6935d640 1 -- 172.21.15.102:0/807611813 shutdown_connections 2024-09-13T17:43:43.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.544+0000 7f7d6935d640 1 -- 172.21.15.102:0/807611813 wait complete. 2024-09-13T17:43:43.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.545+0000 7f7d6935d640 1 Processor -- start 2024-09-13T17:43:43.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.545+0000 7f7d6935d640 1 -- start start 2024-09-13T17:43:43.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.545+0000 7f7d6935d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d64067130 0x7f7d6413cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:43.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.545+0000 7f7d6935d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7d6412db30 0x7f7d6413d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:43.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.545+0000 7f7d6935d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7d6412df00 0x7f7d641163b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:43.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.545+0000 7f7d6935d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6409ea70 con 0x7f7d64067130 2024-09-13T17:43:43.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.545+0000 7f7d6935d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6409a600 con 0x7f7d6412df00 2024-09-13T17:43:43.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.545+0000 7f7d6935d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d64119de0 con 0x7f7d6412db30 2024-09-13T17:43:43.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.545+0000 7f7d62ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d64067130 0x7f7d6413cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:43.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.545+0000 7f7d62ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d64067130 0x7f7d6413cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57516/0 (socket says 172.21.15.102:57516) 2024-09-13T17:43:43.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.545+0000 7f7d62ffd640 1 -- 172.21.15.102:0/2941906635 learned_addr learned my addr 172.21.15.102:0/2941906635 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:43.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.545+0000 7f7d5a7fc640 1 --2- 172.21.15.102:0/2941906635 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7d6412db30 0x7f7d6413d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:43.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.545+0000 7f7d637fe640 1 --2- 172.21.15.102:0/2941906635 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7d6412df00 0x7f7d641163b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:43.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.545+0000 7f7d62ffd640 1 -- 172.21.15.102:0/2941906635 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7d6412db30 msgr2=0x7f7d6413d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:43.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.545+0000 7f7d62ffd640 1 --2- 172.21.15.102:0/2941906635 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7d6412db30 0x7f7d6413d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.545+0000 7f7d62ffd640 1 -- 172.21.15.102:0/2941906635 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7d6412df00 msgr2=0x7f7d641163b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:43.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.545+0000 7f7d62ffd640 1 --2- 172.21.15.102:0/2941906635 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7d6412df00 0x7f7d641163b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.545+0000 7f7d62ffd640 1 -- 172.21.15.102:0/2941906635 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d5c009a20 con 0x7f7d64067130 2024-09-13T17:43:43.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.546+0000 7f7d62ffd640 1 --2- 172.21.15.102:0/2941906635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d64067130 0x7f7d6413cb00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7d5402a6e0 tx=0x7f7d54032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:43.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.546+0000 7f7d5b7fe640 1 -- 172.21.15.102:0/2941906635 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d54031bb0 con 0x7f7d64067130 2024-09-13T17:43:43.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.546+0000 7f7d5b7fe640 1 -- 172.21.15.102:0/2941906635 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d54031d30 con 0x7f7d64067130 2024-09-13T17:43:43.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.546+0000 7f7d6935d640 1 -- 172.21.15.102:0/2941906635 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d54009450 con 0x7f7d64067130 2024-09-13T17:43:43.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.546+0000 7f7d5b7fe640 1 -- 172.21.15.102:0/2941906635 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5403d690 con 0x7f7d64067130 2024-09-13T17:43:43.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.546+0000 7f7d6935d640 1 -- 172.21.15.102:0/2941906635 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d6411a530 con 0x7f7d64067130 2024-09-13T17:43:43.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.547+0000 7f7d5b7fe640 1 -- 172.21.15.102:0/2941906635 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f7d54039040 con 0x7f7d64067130 2024-09-13T17:43:43.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.547+0000 7f7d5b7fe640 1 -- 172.21.15.102:0/2941906635 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f7d54061b80 con 0x7f7d64067130 2024-09-13T17:43:43.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.547+0000 7f7d6935d640 1 -- 172.21.15.102:0/2941906635 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d64064600 con 0x7f7d64067130 2024-09-13T17:43:43.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.551+0000 7f7d5b7fe640 1 -- 172.21.15.102:0/2941906635 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f7d54037080 con 0x7f7d64067130 2024-09-13T17:43:43.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.703+0000 7f7d6935d640 1 -- 172.21.15.102:0/2941906635 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7d64069820 con 0x7f7d64067130 2024-09-13T17:43:43.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.703+0000 7f7d5b7fe640 1 -- 172.21.15.102:0/2941906635 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7d54041020 con 0x7f7d64067130 2024-09-13T17:43:43.706 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-09-13T17:43:43.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.705+0000 7f7d6935d640 1 -- 172.21.15.102:0/2941906635 >> v1:172.21.15.102:6816/34937 conn(0x7f7d44031160 legacy=0x7f7d44033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:43.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.705+0000 7f7d6935d640 1 -- 172.21.15.102:0/2941906635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d64067130 msgr2=0x7f7d6413cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:43.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.705+0000 7f7d6935d640 1 --2- 172.21.15.102:0/2941906635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d64067130 0x7f7d6413cb00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7d5402a6e0 tx=0x7f7d54032000 comp rx=0 tx=0).stop 2024-09-13T17:43:43.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.705+0000 7f7d6935d640 1 -- 172.21.15.102:0/2941906635 shutdown_connections 2024-09-13T17:43:43.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.705+0000 7f7d6935d640 1 --2- 172.21.15.102:0/2941906635 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7d6412df00 0x7f7d641163b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.705+0000 7f7d6935d640 1 --2- 172.21.15.102:0/2941906635 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7d6412db30 0x7f7d6413d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.705+0000 7f7d6935d640 1 --2- 172.21.15.102:0/2941906635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d64067130 0x7f7d6413cb00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.705+0000 7f7d6935d640 1 -- 172.21.15.102:0/2941906635 >> 172.21.15.102:0/2941906635 conn(0x7f7d6405e770 msgr2=0x7f7d64130580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:43.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.705+0000 7f7d6935d640 1 -- 172.21.15.102:0/2941906635 shutdown_connections 2024-09-13T17:43:43.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.705+0000 7f7d6935d640 1 -- 172.21.15.102:0/2941906635 wait complete. 2024-09-13T17:43:43.718 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:43:43.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.813+0000 7fb60484f640 1 Processor -- start 2024-09-13T17:43:43.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.813+0000 7fb60484f640 1 -- start start 2024-09-13T17:43:43.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.813+0000 7fb60484f640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb600096df0 0x7fb6000971c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:43.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.813+0000 7fb60484f640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb600097d30 0x7fb60012d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:43.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.813+0000 7fb60484f640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb60006a850 con 0x7fb600097700 2024-09-13T17:43:43.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.813+0000 7fb60484f640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb600098290 con 0x7fb600096df0 2024-09-13T17:43:43.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.813+0000 7fb60484f640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb600096bf0 con 0x7fb600097d30 2024-09-13T17:43:43.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.814+0000 7fb5fe59c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb600096df0 0x7fb6000971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:43.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.814+0000 7fb5fed9d640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fb600097700 0x7fb60012d7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36316/0 (socket says 172.21.15.102:36316) 2024-09-13T17:43:43.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.814+0000 7fb5fed9d640 1 -- 172.21.15.102:0/2696215263 learned_addr learned my addr 172.21.15.102:0/2696215263 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:43.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.814+0000 7fb5fdd9b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb600097d30 0x7fb60012d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:43.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.814+0000 7fb5fdd9b640 1 -- 172.21.15.102:0/2696215263 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb600096df0 msgr2=0x7fb6000971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:43.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.814+0000 7fb5fdd9b640 1 --2- 172.21.15.102:0/2696215263 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb600096df0 0x7fb6000971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.814+0000 7fb5fd59a640 1 -- 172.21.15.102:0/2696215263 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1557720819 0 0) 0x7fb60006a850 con 0x7fb600097700 2024-09-13T17:43:43.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.814+0000 7fb5fdd9b640 1 -- 172.21.15.102:0/2696215263 >> v1:172.21.15.102:6789/0 conn(0x7fb600097700 legacy=0x7fb60012d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:43.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.814+0000 7fb5fdd9b640 1 -- 172.21.15.102:0/2696215263 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb60006a590 con 0x7fb600097d30 2024-09-13T17:43:43.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.814+0000 7fb5fe59c640 1 --2- 172.21.15.102:0/2696215263 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb600096df0 0x7fb6000971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:43.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.815+0000 7fb5fdd9b640 1 --2- 172.21.15.102:0/2696215263 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb600097d30 0x7fb60012d280 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40098b0 tx=0x7fb5f40090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dc3bcbf842c571a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:43.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.815+0000 7fb5fd59a640 1 -- 172.21.15.102:0/2696215263 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f402acc0 con 0x7fb600097d30 2024-09-13T17:43:43.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.815+0000 7fb5fd59a640 1 -- 172.21.15.102:0/2696215263 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f402ae20 con 0x7fb600097d30 2024-09-13T17:43:43.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.815+0000 7fb5fd59a640 1 -- 172.21.15.102:0/2696215263 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4032670 con 0x7fb600097d30 2024-09-13T17:43:43.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.815+0000 7fb60484f640 1 -- 172.21.15.102:0/2696215263 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb600097d30 msgr2=0x7fb60012d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:43.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.815+0000 7fb60484f640 1 --2- 172.21.15.102:0/2696215263 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb600097d30 0x7fb60012d280 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40098b0 tx=0x7fb5f40090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.815+0000 7fb60484f640 1 -- 172.21.15.102:0/2696215263 shutdown_connections 2024-09-13T17:43:43.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.815+0000 7fb60484f640 1 --2- 172.21.15.102:0/2696215263 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb600097d30 0x7fb60012d280 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.815+0000 7fb60484f640 1 --2- 172.21.15.102:0/2696215263 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb600096df0 0x7fb6000971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.815+0000 7fb60484f640 1 -- 172.21.15.102:0/2696215263 >> 172.21.15.102:0/2696215263 conn(0x7fb60009e720 msgr2=0x7fb60009eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:43.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.815+0000 7fb60484f640 1 -- 172.21.15.102:0/2696215263 shutdown_connections 2024-09-13T17:43:43.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.815+0000 7fb60484f640 1 -- 172.21.15.102:0/2696215263 wait complete. 2024-09-13T17:43:43.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.816+0000 7fb60484f640 1 Processor -- start 2024-09-13T17:43:43.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.816+0000 7fb60484f640 1 -- start start 2024-09-13T17:43:43.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.817+0000 7fb60484f640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb600096df0 0x7fb600119b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:43.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.817+0000 7fb60484f640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb600097700 0x7fb60011a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:43.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.817+0000 7fb60484f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb600097d30 0x7fb60013c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:43.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.817+0000 7fb60484f640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb600119910 con 0x7fb600097d30 2024-09-13T17:43:43.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.817+0000 7fb60484f640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb60006bc90 con 0x7fb600096df0 2024-09-13T17:43:43.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.817+0000 7fb60484f640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb600069df0 con 0x7fb600097700 2024-09-13T17:43:43.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.817+0000 7fb5fe59c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb600096df0 0x7fb600119b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:43.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.817+0000 7fb5fdd9b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb600097700 0x7fb60011a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:43.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.817+0000 7fb5fed9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb600097d30 0x7fb60013c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:43.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.817+0000 7fb5fe59c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb600096df0 0x7fb600119b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:50740/0 (socket says 172.21.15.102:50740) 2024-09-13T17:43:43.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.817+0000 7fb5fdd9b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb600097700 0x7fb60011a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:43208/0 (socket says 172.21.15.102:43208) 2024-09-13T17:43:43.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.817+0000 7fb5fe59c640 1 -- 172.21.15.102:0/466098443 learned_addr learned my addr 172.21.15.102:0/466098443 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:43.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.817+0000 7fb5fe59c640 1 -- 172.21.15.102:0/466098443 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb600097700 msgr2=0x7fb60011a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:43.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.817+0000 7fb5fe59c640 1 --2- 172.21.15.102:0/466098443 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb600097700 0x7fb60011a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.817+0000 7fb5fe59c640 1 -- 172.21.15.102:0/466098443 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb600097d30 msgr2=0x7fb60013c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:43.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.817+0000 7fb5fe59c640 1 --2- 172.21.15.102:0/466098443 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb600097d30 0x7fb60013c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.817+0000 7fb5fe59c640 1 -- 172.21.15.102:0/466098443 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f0009390 con 0x7fb600096df0 2024-09-13T17:43:43.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.817+0000 7fb5fe59c640 1 --2- 172.21.15.102:0/466098443 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb600096df0 0x7fb600119b40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000ec80 tx=0x7fb5f00090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:43.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.817+0000 7fb5ee7fc640 1 -- 172.21.15.102:0/466098443 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f000eec0 con 0x7fb600096df0 2024-09-13T17:43:43.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.817+0000 7fb5ee7fc640 1 -- 172.21.15.102:0/466098443 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f000cbc0 con 0x7fb600096df0 2024-09-13T17:43:43.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.817+0000 7fb60484f640 1 -- 172.21.15.102:0/466098443 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f4009390 con 0x7fb600096df0 2024-09-13T17:43:43.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.817+0000 7fb5ee7fc640 1 -- 172.21.15.102:0/466098443 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0013470 con 0x7fb600096df0 2024-09-13T17:43:43.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.817+0000 7fb60484f640 1 -- 172.21.15.102:0/466098443 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb60011d640 con 0x7fb600096df0 2024-09-13T17:43:43.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.818+0000 7fb5ee7fc640 1 -- 172.21.15.102:0/466098443 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fb5f00137e0 con 0x7fb600096df0 2024-09-13T17:43:43.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.818+0000 7fb60484f640 1 -- 172.21.15.102:0/466098443 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb600064000 con 0x7fb600096df0 2024-09-13T17:43:43.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.818+0000 7fb5ee7fc640 1 -- 172.21.15.102:0/466098443 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7fb5f00424e0 con 0x7fb600096df0 2024-09-13T17:43:43.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.823+0000 7fb5ee7fc640 1 -- 172.21.15.102:0/466098443 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fb5f0023ad0 con 0x7fb600096df0 2024-09-13T17:43:43.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.972+0000 7fb60484f640 1 -- 172.21.15.102:0/466098443 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb60011c8b0 con 0x7fb600096df0 2024-09-13T17:43:43.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.973+0000 7fb5ee7fc640 1 -- 172.21.15.102:0/466098443 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb5f00210a0 con 0x7fb600096df0 2024-09-13T17:43:43.975 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-09-13T17:43:43.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.975+0000 7fb60484f640 1 -- 172.21.15.102:0/466098443 >> v1:172.21.15.102:6816/34937 conn(0x7fb5e0031160 legacy=0x7fb5e0033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:43.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.975+0000 7fb60484f640 1 -- 172.21.15.102:0/466098443 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb600096df0 msgr2=0x7fb600119b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:43.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.975+0000 7fb60484f640 1 --2- 172.21.15.102:0/466098443 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb600096df0 0x7fb600119b40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000ec80 tx=0x7fb5f00090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.975+0000 7fb60484f640 1 -- 172.21.15.102:0/466098443 shutdown_connections 2024-09-13T17:43:43.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.975+0000 7fb60484f640 1 --2- 172.21.15.102:0/466098443 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb600097d30 0x7fb60013c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.975+0000 7fb60484f640 1 --2- 172.21.15.102:0/466098443 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb600097700 0x7fb60011a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.975+0000 7fb60484f640 1 --2- 172.21.15.102:0/466098443 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb600096df0 0x7fb600119b40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:43.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.975+0000 7fb60484f640 1 -- 172.21.15.102:0/466098443 >> 172.21.15.102:0/466098443 conn(0x7fb60009e720 msgr2=0x7fb60009c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:43.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.975+0000 7fb60484f640 1 -- 172.21.15.102:0/466098443 shutdown_connections 2024-09-13T17:43:43.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:43.975+0000 7fb60484f640 1 -- 172.21.15.102:0/466098443 wait complete. 2024-09-13T17:43:43.987 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:43:44.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.077+0000 7f528e434640 1 Processor -- start 2024-09-13T17:43:44.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.077+0000 7f528e434640 1 -- start start 2024-09-13T17:43:44.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.078+0000 7f528e434640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5288128e10 0x7f5288130b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:44.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.078+0000 7f528e434640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f52881310b0 0x7f5288133490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:44.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.078+0000 7f528e434640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5288097be0 con 0x7f5288128330 2024-09-13T17:43:44.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.078+0000 7f528e434640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5288129200 con 0x7f5288128e10 2024-09-13T17:43:44.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.078+0000 7f528e434640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5288098e70 con 0x7f52881310b0 2024-09-13T17:43:44.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.078+0000 7f528c9d1640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f52881310b0 0x7f5288133490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:44.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.078+0000 7f52877fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5288128e10 0x7f5288130b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:44.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.078+0000 7f5287fff640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f5288128330 0x7f5288128700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36328/0 (socket says 172.21.15.102:36328) 2024-09-13T17:43:44.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.078+0000 7f5287fff640 1 -- 172.21.15.102:0/865613699 learned_addr learned my addr 172.21.15.102:0/865613699 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:44.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.078+0000 7f52877fe640 1 -- 172.21.15.102:0/865613699 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f52881310b0 msgr2=0x7f5288133490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:44.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.078+0000 7f52877fe640 1 --2- 172.21.15.102:0/865613699 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f52881310b0 0x7f5288133490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.078+0000 7f52877fe640 1 -- 172.21.15.102:0/865613699 >> v1:172.21.15.102:6789/0 conn(0x7f5288128330 legacy=0x7f5288128700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:44.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.078+0000 7f5286ffd640 1 -- 172.21.15.102:0/865613699 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2656630329 0 0) 0x7f5288097be0 con 0x7f5288128330 2024-09-13T17:43:44.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.078+0000 7f52877fe640 1 -- 172.21.15.102:0/865613699 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5288098310 con 0x7f5288128e10 2024-09-13T17:43:44.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.078+0000 7f528c9d1640 1 --2- 172.21.15.102:0/865613699 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f52881310b0 0x7f5288133490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:44.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.079+0000 7f52877fe640 1 --2- 172.21.15.102:0/865613699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5288128e10 0x7f5288130b70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5270009850 tx=0x7f52700090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a65af0029f23460 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:44.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.079+0000 7f5286ffd640 1 -- 172.21.15.102:0/865613699 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527002ad80 con 0x7f5288128e10 2024-09-13T17:43:44.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.079+0000 7f5286ffd640 1 -- 172.21.15.102:0/865613699 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5270032440 con 0x7f5288128e10 2024-09-13T17:43:44.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.079+0000 7f5286ffd640 1 -- 172.21.15.102:0/865613699 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52700327e0 con 0x7f5288128e10 2024-09-13T17:43:44.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.080+0000 7f528e434640 1 -- 172.21.15.102:0/865613699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5288128e10 msgr2=0x7f5288130b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:44.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.080+0000 7f528e434640 1 --2- 172.21.15.102:0/865613699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5288128e10 0x7f5288130b70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5270009850 tx=0x7f52700090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.080+0000 7f528e434640 1 -- 172.21.15.102:0/865613699 shutdown_connections 2024-09-13T17:43:44.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.080+0000 7f528e434640 1 --2- 172.21.15.102:0/865613699 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f52881310b0 0x7f5288133490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.080+0000 7f528e434640 1 --2- 172.21.15.102:0/865613699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5288128e10 0x7f5288130b70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.080+0000 7f528e434640 1 -- 172.21.15.102:0/865613699 >> 172.21.15.102:0/865613699 conn(0x7f528809e820 msgr2=0x7f528809ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:44.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.080+0000 7f528e434640 1 -- 172.21.15.102:0/865613699 shutdown_connections 2024-09-13T17:43:44.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.080+0000 7f528e434640 1 -- 172.21.15.102:0/865613699 wait complete. 2024-09-13T17:43:44.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.081+0000 7f528e434640 1 Processor -- start 2024-09-13T17:43:44.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.081+0000 7f528e434640 1 -- start start 2024-09-13T17:43:44.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.081+0000 7f528e434640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5288128330 0x7f528813ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:44.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.081+0000 7f528e434640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5288128e10 0x7f528813cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:44.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.081+0000 7f528e434640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52881310b0 0x7f528806b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:44.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.081+0000 7f528e434640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528806c7d0 con 0x7f52881310b0 2024-09-13T17:43:44.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.081+0000 7f528e434640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f528806d030 con 0x7f5288128330 2024-09-13T17:43:44.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.081+0000 7f528e434640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5288130b90 con 0x7f5288128e10 2024-09-13T17:43:44.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.081+0000 7f5287fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5288128330 0x7f528813ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:44.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.081+0000 7f52877fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5288128e10 0x7f528813cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:44.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.081+0000 7f528c9d1640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52881310b0 0x7f528806b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:44.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.081+0000 7f5287fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5288128330 0x7f528813ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:50762/0 (socket says 172.21.15.102:50762) 2024-09-13T17:43:44.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.081+0000 7f52877fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5288128e10 0x7f528813cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:43210/0 (socket says 172.21.15.102:43210) 2024-09-13T17:43:44.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.081+0000 7f5287fff640 1 -- 172.21.15.102:0/3473921963 learned_addr learned my addr 172.21.15.102:0/3473921963 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:44.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.082+0000 7f528c9d1640 1 -- 172.21.15.102:0/3473921963 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5288128e10 msgr2=0x7f528813cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:44.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.082+0000 7f528c9d1640 1 --2- 172.21.15.102:0/3473921963 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5288128e10 0x7f528813cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.082+0000 7f528c9d1640 1 -- 172.21.15.102:0/3473921963 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5288128330 msgr2=0x7f528813ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:44.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.082+0000 7f528c9d1640 1 --2- 172.21.15.102:0/3473921963 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5288128330 0x7f528813ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.082+0000 7f528c9d1640 1 -- 172.21.15.102:0/3473921963 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5280009a20 con 0x7f52881310b0 2024-09-13T17:43:44.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.082+0000 7f5287fff640 1 --2- 172.21.15.102:0/3473921963 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5288128330 0x7f528813ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:44.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.082+0000 7f528c9d1640 1 --2- 172.21.15.102:0/3473921963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52881310b0 0x7f528806b360 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f528000b430 tx=0x7f528000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:44.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.082+0000 7f527ffff640 1 -- 172.21.15.102:0/3473921963 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5280015070 con 0x7f52881310b0 2024-09-13T17:43:44.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.082+0000 7f527ffff640 1 -- 172.21.15.102:0/3473921963 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5280019410 con 0x7f52881310b0 2024-09-13T17:43:44.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.082+0000 7f528e434640 1 -- 172.21.15.102:0/3473921963 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5270009390 con 0x7f52881310b0 2024-09-13T17:43:44.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.082+0000 7f527ffff640 1 -- 172.21.15.102:0/3473921963 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52800197b0 con 0x7f52881310b0 2024-09-13T17:43:44.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.082+0000 7f528e434640 1 -- 172.21.15.102:0/3473921963 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f528813d490 con 0x7f52881310b0 2024-09-13T17:43:44.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.083+0000 7f527ffff640 1 -- 172.21.15.102:0/3473921963 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f5280003020 con 0x7f52881310b0 2024-09-13T17:43:44.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.083+0000 7f527ffff640 1 -- 172.21.15.102:0/3473921963 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f5280046570 con 0x7f52881310b0 2024-09-13T17:43:44.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.083+0000 7f528e434640 1 -- 172.21.15.102:0/3473921963 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5288064000 con 0x7f52881310b0 2024-09-13T17:43:44.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.087+0000 7f527ffff640 1 -- 172.21.15.102:0/3473921963 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f528001d070 con 0x7f52881310b0 2024-09-13T17:43:44.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.241+0000 7f528e434640 1 -- 172.21.15.102:0/3473921963 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5288067690 con 0x7f52881310b0 2024-09-13T17:43:44.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.241+0000 7f527ffff640 1 -- 172.21.15.102:0/3473921963 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+11660 (secure 0 0 0) 0x7f528001f4e0 con 0x7f52881310b0 2024-09-13T17:43:44.244 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:43:44.244 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":23,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:42:48.385896+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:04.672265+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:05.583134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.17","mappings":[{"from":3,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]},{"pgid":"4.25","mappings":[{"from":3,"to":2}]},{"pgid":"4.33","mappings":[{"from":3,"to":0}]},{"pgid":"4.3a","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:43:44.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.243+0000 7f528e434640 1 -- 172.21.15.102:0/3473921963 >> v1:172.21.15.102:6816/34937 conn(0x7f5268031160 legacy=0x7f5268033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:44.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.243+0000 7f528e434640 1 -- 172.21.15.102:0/3473921963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52881310b0 msgr2=0x7f528806b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:44.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.243+0000 7f528e434640 1 --2- 172.21.15.102:0/3473921963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52881310b0 0x7f528806b360 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f528000b430 tx=0x7f528000b040 comp rx=0 tx=0).stop 2024-09-13T17:43:44.246 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.243+0000 7f528e434640 1 -- 172.21.15.102:0/3473921963 shutdown_connections 2024-09-13T17:43:44.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.243+0000 7f528e434640 1 --2- 172.21.15.102:0/3473921963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52881310b0 0x7f528806b360 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.243+0000 7f528e434640 1 --2- 172.21.15.102:0/3473921963 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5288128e10 0x7f528813cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.243+0000 7f528e434640 1 --2- 172.21.15.102:0/3473921963 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5288128330 0x7f528813ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.243+0000 7f528e434640 1 -- 172.21.15.102:0/3473921963 >> 172.21.15.102:0/3473921963 conn(0x7f528809e820 msgr2=0x7f528809c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:44.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.243+0000 7f528e434640 1 -- 172.21.15.102:0/3473921963 shutdown_connections 2024-09-13T17:43:44.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.244+0000 7f528e434640 1 -- 172.21.15.102:0/3473921963 wait complete. 2024-09-13T17:43:44.257 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:54.442612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T17:41:59.076782+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 16, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:42:04.672265+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:42:05.583134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:43:44.257 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:43:44.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.351+0000 7f9bbdb5d640 1 Processor -- start 2024-09-13T17:43:44.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.351+0000 7f9bbdb5d640 1 -- start start 2024-09-13T17:43:44.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.352+0000 7f9bbdb5d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9bb809df40 0x7f9bb8134960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:44.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.352+0000 7f9bbdb5d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9bb8134ea0 0x7f9bb8137280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:44.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.352+0000 7f9bbdb5d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bb8096b10 con 0x7f9bb80689d0 2024-09-13T17:43:44.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.352+0000 7f9bbdb5d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bb8068da0 con 0x7f9bb809df40 2024-09-13T17:43:44.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.352+0000 7f9bbdb5d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb8096db0 con 0x7f9bb8134ea0 2024-09-13T17:43:44.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.352+0000 7f9bb77fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f9bb80689d0 0x7f9bb809d830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36344/0 (socket says 172.21.15.102:36344) 2024-09-13T17:43:44.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.352+0000 7f9bb77fe640 1 -- 172.21.15.102:0/3429571459 learned_addr learned my addr 172.21.15.102:0/3429571459 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:44.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.352+0000 7f9bb6ffd640 1 --2- 172.21.15.102:0/3429571459 >> v2:172.21.15.102:3300/0 conn(0x7f9bb809df40 0x7f9bb8134960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:44.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.352+0000 7f9bb7fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9bb8134ea0 0x7f9bb8137280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:44.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.352+0000 7f9bb6ffd640 1 -- 172.21.15.102:0/3429571459 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9bb8134ea0 msgr2=0x7f9bb8137280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:44.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.352+0000 7f9bb6ffd640 1 --2- 172.21.15.102:0/3429571459 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9bb8134ea0 0x7f9bb8137280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.352+0000 7f9bb6ffd640 1 -- 172.21.15.102:0/3429571459 >> v1:172.21.15.102:6789/0 conn(0x7f9bb80689d0 legacy=0x7f9bb809d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:44.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.352+0000 7f9bb67fc640 1 -- 172.21.15.102:0/3429571459 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2128652328 0 0) 0x7f9bb8096b10 con 0x7f9bb80689d0 2024-09-13T17:43:44.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.352+0000 7f9bb6ffd640 1 -- 172.21.15.102:0/3429571459 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb80696e0 con 0x7f9bb809df40 2024-09-13T17:43:44.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.352+0000 7f9bb7fff640 1 --2- 172.21.15.102:0/3429571459 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9bb8134ea0 0x7f9bb8137280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:44.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.353+0000 7f9bb6ffd640 1 --2- 172.21.15.102:0/3429571459 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9bb809df40 0x7f9bb8134960 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9bac009850 tx=0x7f9bac0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e33b84d7244b899 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:44.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.353+0000 7f9bb67fc640 1 -- 172.21.15.102:0/3429571459 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac02ac80 con 0x7f9bb809df40 2024-09-13T17:43:44.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.353+0000 7f9bb67fc640 1 -- 172.21.15.102:0/3429571459 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bac02ade0 con 0x7f9bb809df40 2024-09-13T17:43:44.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.353+0000 7f9bb67fc640 1 -- 172.21.15.102:0/3429571459 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac032670 con 0x7f9bb809df40 2024-09-13T17:43:44.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.354+0000 7f9bbdb5d640 1 -- 172.21.15.102:0/3429571459 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9bb809df40 msgr2=0x7f9bb8134960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:44.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.354+0000 7f9bbdb5d640 1 --2- 172.21.15.102:0/3429571459 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9bb809df40 0x7f9bb8134960 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9bac009850 tx=0x7f9bac0090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.354+0000 7f9bbdb5d640 1 -- 172.21.15.102:0/3429571459 shutdown_connections 2024-09-13T17:43:44.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.354+0000 7f9bbdb5d640 1 --2- 172.21.15.102:0/3429571459 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9bb8134ea0 0x7f9bb8137280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.354+0000 7f9bbdb5d640 1 --2- 172.21.15.102:0/3429571459 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9bb809df40 0x7f9bb8134960 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.354+0000 7f9bbdb5d640 1 -- 172.21.15.102:0/3429571459 >> 172.21.15.102:0/3429571459 conn(0x7f9bb806f320 msgr2=0x7f9bb80a2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:44.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.354+0000 7f9bbdb5d640 1 -- 172.21.15.102:0/3429571459 shutdown_connections 2024-09-13T17:43:44.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.354+0000 7f9bbdb5d640 1 -- 172.21.15.102:0/3429571459 wait complete. 2024-09-13T17:43:44.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.355+0000 7f9bbdb5d640 1 Processor -- start 2024-09-13T17:43:44.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.355+0000 7f9bbdb5d640 1 -- start start 2024-09-13T17:43:44.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.355+0000 7f9bbdb5d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9bb80689d0 0x7f9bb811e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:44.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.355+0000 7f9bbdb5d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9bb809df40 0x7f9bb811a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:44.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.355+0000 7f9bbdb5d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9bb8134ea0 0x7f9bb811a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:44.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.355+0000 7f9bbdb5d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb811c090 con 0x7f9bb8134ea0 2024-09-13T17:43:44.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.355+0000 7f9bbdb5d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb811d470 con 0x7f9bb80689d0 2024-09-13T17:43:44.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.355+0000 7f9bbdb5d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb8130b40 con 0x7f9bb809df40 2024-09-13T17:43:44.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.355+0000 7f9bb77fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9bb80689d0 0x7f9bb811e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:44.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.355+0000 7f9bb7fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9bb8134ea0 0x7f9bb811a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:44.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.355+0000 7f9bb6ffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9bb809df40 0x7f9bb811a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:44.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.355+0000 7f9bb7fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9bb8134ea0 0x7f9bb811a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57556/0 (socket says 172.21.15.102:57556) 2024-09-13T17:43:44.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.355+0000 7f9bb6ffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9bb809df40 0x7f9bb811a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:43224/0 (socket says 172.21.15.102:43224) 2024-09-13T17:43:44.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.355+0000 7f9bb77fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9bb80689d0 0x7f9bb811e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:50770/0 (socket says 172.21.15.102:50770) 2024-09-13T17:43:44.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.355+0000 7f9bb7fff640 1 -- 172.21.15.102:0/384184908 learned_addr learned my addr 172.21.15.102:0/384184908 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:44.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.355+0000 7f9bb7fff640 1 -- 172.21.15.102:0/384184908 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9bb809df40 msgr2=0x7f9bb811a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:44.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.355+0000 7f9bb7fff640 1 --2- 172.21.15.102:0/384184908 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9bb809df40 0x7f9bb811a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.355+0000 7f9bb7fff640 1 -- 172.21.15.102:0/384184908 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9bb80689d0 msgr2=0x7f9bb811e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:44.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.355+0000 7f9bb7fff640 1 --2- 172.21.15.102:0/384184908 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9bb80689d0 0x7f9bb811e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.355+0000 7f9bb7fff640 1 -- 172.21.15.102:0/384184908 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb0009a20 con 0x7f9bb8134ea0 2024-09-13T17:43:44.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.355+0000 7f9bb77fe640 1 --2- 172.21.15.102:0/384184908 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9bb80689d0 0x7f9bb811e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:44.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.356+0000 7f9bb7fff640 1 --2- 172.21.15.102:0/384184908 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9bb8134ea0 0x7f9bb811a910 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0009f90 tx=0x7f9bb000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:44.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.356+0000 7f9b9f7fe640 1 -- 172.21.15.102:0/384184908 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb000ce70 con 0x7f9bb8134ea0 2024-09-13T17:43:44.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.356+0000 7f9b9f7fe640 1 -- 172.21.15.102:0/384184908 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb000f040 con 0x7f9bb8134ea0 2024-09-13T17:43:44.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.356+0000 7f9bbdb5d640 1 -- 172.21.15.102:0/384184908 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bac009390 con 0x7f9bb8134ea0 2024-09-13T17:43:44.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.356+0000 7f9b9f7fe640 1 -- 172.21.15.102:0/384184908 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb00146f0 con 0x7f9bb8134ea0 2024-09-13T17:43:44.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.356+0000 7f9bbdb5d640 1 -- 172.21.15.102:0/384184908 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bb811d810 con 0x7f9bb8134ea0 2024-09-13T17:43:44.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.356+0000 7f9b9f7fe640 1 -- 172.21.15.102:0/384184908 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f9bb0014a60 con 0x7f9bb8134ea0 2024-09-13T17:43:44.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.357+0000 7f9b9f7fe640 1 -- 172.21.15.102:0/384184908 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f9bb0042f20 con 0x7f9bb8134ea0 2024-09-13T17:43:44.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.357+0000 7f9bbdb5d640 1 -- 172.21.15.102:0/384184908 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bb8064000 con 0x7f9bb8134ea0 2024-09-13T17:43:44.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.361+0000 7f9b9f7fe640 1 -- 172.21.15.102:0/384184908 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f9bb0018070 con 0x7f9bb8134ea0 2024-09-13T17:43:44.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.513+0000 7f9bbdb5d640 1 -- 172.21.15.102:0/384184908 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9bb80a39c0 con 0x7f9bb8134ea0 2024-09-13T17:43:44.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.514+0000 7f9b9f7fe640 1 -- 172.21.15.102:0/384184908 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9bb0024530 con 0x7f9bb8134ea0 2024-09-13T17:43:44.516 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-09-13T17:43:44.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.517+0000 7f9bbdb5d640 1 -- 172.21.15.102:0/384184908 >> v1:172.21.15.102:6816/34937 conn(0x7f9ba4031160 legacy=0x7f9ba4033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:44.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.517+0000 7f9bbdb5d640 1 -- 172.21.15.102:0/384184908 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9bb8134ea0 msgr2=0x7f9bb811a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:44.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.517+0000 7f9bbdb5d640 1 --2- 172.21.15.102:0/384184908 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9bb8134ea0 0x7f9bb811a910 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0009f90 tx=0x7f9bb000b040 comp rx=0 tx=0).stop 2024-09-13T17:43:44.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.517+0000 7f9bbdb5d640 1 -- 172.21.15.102:0/384184908 shutdown_connections 2024-09-13T17:43:44.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.517+0000 7f9bbdb5d640 1 --2- 172.21.15.102:0/384184908 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9bb8134ea0 0x7f9bb811a910 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.517+0000 7f9bbdb5d640 1 --2- 172.21.15.102:0/384184908 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9bb809df40 0x7f9bb811a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.517+0000 7f9bbdb5d640 1 --2- 172.21.15.102:0/384184908 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9bb80689d0 0x7f9bb811e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.517+0000 7f9bbdb5d640 1 -- 172.21.15.102:0/384184908 >> 172.21.15.102:0/384184908 conn(0x7f9bb806f320 msgr2=0x7f9bb80a18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:44.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.517+0000 7f9bbdb5d640 1 -- 172.21.15.102:0/384184908 shutdown_connections 2024-09-13T17:43:44.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.517+0000 7f9bbdb5d640 1 -- 172.21.15.102:0/384184908 wait complete. 2024-09-13T17:43:44.530 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-13T17:43:44.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.624+0000 7fa3eeb9d640 1 Processor -- start 2024-09-13T17:43:44.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.624+0000 7fa3eeb9d640 1 -- start start 2024-09-13T17:43:44.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.624+0000 7fa3eeb9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa3e8096df0 0x7fa3e80971c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:44.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.624+0000 7fa3eeb9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa3e8097d30 0x7fa3e812d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:44.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.624+0000 7fa3eeb9d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa3e8097700 0x7fa3e812d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:44.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.624+0000 7fa3eeb9d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3e8098290 con 0x7fa3e8097700 2024-09-13T17:43:44.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.624+0000 7fa3eeb9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e806a850 con 0x7fa3e8097d30 2024-09-13T17:43:44.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.624+0000 7fa3eeb9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e8096bf0 con 0x7fa3e8096df0 2024-09-13T17:43:44.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.624+0000 7fa3ec939640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa3e8096df0 0x7fa3e80971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:44.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.624+0000 7fa3ed13a640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa3e8097700 0x7fa3e812d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:44.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.625+0000 7fa3ec939640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa3e8096df0 0x7fa3e80971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:43232/0 (socket says 172.21.15.102:43232) 2024-09-13T17:43:44.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.625+0000 7fa3ec939640 1 -- 172.21.15.102:0/2724121019 learned_addr learned my addr 172.21.15.102:0/2724121019 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:44.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.625+0000 7fa3ed13a640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa3e8097700 0x7fa3e812d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57558/0 (socket says 172.21.15.102:57558) 2024-09-13T17:43:44.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.625+0000 7fa3dffff640 1 --2- 172.21.15.102:0/2724121019 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa3e8097d30 0x7fa3e812d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:44.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.625+0000 7fa3dffff640 1 -- 172.21.15.102:0/2724121019 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa3e8096df0 msgr2=0x7fa3e80971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:44.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.625+0000 7fa3dffff640 1 --2- 172.21.15.102:0/2724121019 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa3e8096df0 0x7fa3e80971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.625+0000 7fa3dffff640 1 -- 172.21.15.102:0/2724121019 >> v2:172.21.15.102:3300/0 conn(0x7fa3e8097700 msgr2=0x7fa3e812d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:44.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.625+0000 7fa3dffff640 1 --2- 172.21.15.102:0/2724121019 >> v2:172.21.15.102:3300/0 conn(0x7fa3e8097700 0x7fa3e812d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.625+0000 7fa3dffff640 1 -- 172.21.15.102:0/2724121019 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e806a590 con 0x7fa3e8097d30 2024-09-13T17:43:44.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.625+0000 7fa3ec939640 1 --2- 172.21.15.102:0/2724121019 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa3e8096df0 0x7fa3e80971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:44.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.625+0000 7fa3dffff640 1 --2- 172.21.15.102:0/2724121019 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa3e8097d30 0x7fa3e812d280 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8009880 tx=0x7fa3d80090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f9c5460a262004 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:44.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.626+0000 7fa3df7fe640 1 -- 172.21.15.102:0/2724121019 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d802ac40 con 0x7fa3e8097d30 2024-09-13T17:43:44.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.626+0000 7fa3df7fe640 1 -- 172.21.15.102:0/2724121019 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d802ada0 con 0x7fa3e8097d30 2024-09-13T17:43:44.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.626+0000 7fa3df7fe640 1 -- 172.21.15.102:0/2724121019 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d8032670 con 0x7fa3e8097d30 2024-09-13T17:43:44.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.626+0000 7fa3eeb9d640 1 -- 172.21.15.102:0/2724121019 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa3e8097d30 msgr2=0x7fa3e812d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:44.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.626+0000 7fa3eeb9d640 1 --2- 172.21.15.102:0/2724121019 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa3e8097d30 0x7fa3e812d280 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8009880 tx=0x7fa3d80090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.626+0000 7fa3eeb9d640 1 -- 172.21.15.102:0/2724121019 shutdown_connections 2024-09-13T17:43:44.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.626+0000 7fa3eeb9d640 1 --2- 172.21.15.102:0/2724121019 >> v2:172.21.15.102:3300/0 conn(0x7fa3e8097700 0x7fa3e812d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.626+0000 7fa3eeb9d640 1 --2- 172.21.15.102:0/2724121019 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa3e8097d30 0x7fa3e812d280 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.626+0000 7fa3eeb9d640 1 --2- 172.21.15.102:0/2724121019 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa3e8096df0 0x7fa3e80971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.626+0000 7fa3eeb9d640 1 -- 172.21.15.102:0/2724121019 >> 172.21.15.102:0/2724121019 conn(0x7fa3e809e720 msgr2=0x7fa3e809eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:44.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.626+0000 7fa3eeb9d640 1 -- 172.21.15.102:0/2724121019 shutdown_connections 2024-09-13T17:43:44.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.626+0000 7fa3eeb9d640 1 -- 172.21.15.102:0/2724121019 wait complete. 2024-09-13T17:43:44.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.627+0000 7fa3eeb9d640 1 Processor -- start 2024-09-13T17:43:44.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.627+0000 7fa3eeb9d640 1 -- start start 2024-09-13T17:43:44.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.628+0000 7fa3eeb9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa3e8096df0 0x7fa3e811a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:44.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.628+0000 7fa3eeb9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa3e8097700 0x7fa3e8116bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:44.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.628+0000 7fa3eeb9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3e8097d30 0x7fa3e8117110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:44.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.628+0000 7fa3eeb9d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e81169d0 con 0x7fa3e8097d30 2024-09-13T17:43:44.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.628+0000 7fa3eeb9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e806be00 con 0x7fa3e8097700 2024-09-13T17:43:44.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.628+0000 7fa3eeb9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e8069df0 con 0x7fa3e8096df0 2024-09-13T17:43:44.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.628+0000 7fa3ec939640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa3e8096df0 0x7fa3e811a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:44.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.628+0000 7fa3ed13a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3e8097d30 0x7fa3e8117110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:44.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.628+0000 7fa3ec939640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa3e8096df0 0x7fa3e811a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:43246/0 (socket says 172.21.15.102:43246) 2024-09-13T17:43:44.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.628+0000 7fa3ec939640 1 -- 172.21.15.102:0/1987877079 learned_addr learned my addr 172.21.15.102:0/1987877079 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:44.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.628+0000 7fa3dffff640 1 --2- 172.21.15.102:0/1987877079 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa3e8097700 0x7fa3e8116bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:44.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.628+0000 7fa3ed13a640 1 -- 172.21.15.102:0/1987877079 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa3e8096df0 msgr2=0x7fa3e811a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:44.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.628+0000 7fa3ed13a640 1 --2- 172.21.15.102:0/1987877079 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa3e8096df0 0x7fa3e811a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.628+0000 7fa3ed13a640 1 -- 172.21.15.102:0/1987877079 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa3e8097700 msgr2=0x7fa3e8116bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:44.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.628+0000 7fa3ed13a640 1 --2- 172.21.15.102:0/1987877079 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa3e8097700 0x7fa3e8116bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.628+0000 7fa3ed13a640 1 -- 172.21.15.102:0/1987877079 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e4009450 con 0x7fa3e8097d30 2024-09-13T17:43:44.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.628+0000 7fa3ec939640 1 --2- 172.21.15.102:0/1987877079 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa3e8096df0 0x7fa3e811a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:44.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.628+0000 7fa3dffff640 1 --2- 172.21.15.102:0/1987877079 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa3e8097700 0x7fa3e8116bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:44.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.628+0000 7fa3ed13a640 1 --2- 172.21.15.102:0/1987877079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3e8097d30 0x7fa3e8117110 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009e50 tx=0x7fa3e000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:44.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.628+0000 7fa3dcff9640 1 -- 172.21.15.102:0/1987877079 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e000cd40 con 0x7fa3e8097d30 2024-09-13T17:43:44.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.628+0000 7fa3dcff9640 1 -- 172.21.15.102:0/1987877079 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e000f040 con 0x7fa3e8097d30 2024-09-13T17:43:44.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.628+0000 7fa3eeb9d640 1 -- 172.21.15.102:0/1987877079 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d8009390 con 0x7fa3e8097d30 2024-09-13T17:43:44.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.628+0000 7fa3dcff9640 1 -- 172.21.15.102:0/1987877079 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e00146d0 con 0x7fa3e8097d30 2024-09-13T17:43:44.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.628+0000 7fa3eeb9d640 1 -- 172.21.15.102:0/1987877079 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3e8117650 con 0x7fa3e8097d30 2024-09-13T17:43:44.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.629+0000 7fa3dcff9640 1 -- 172.21.15.102:0/1987877079 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fa3e0014a40 con 0x7fa3e8097d30 2024-09-13T17:43:44.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.629+0000 7fa3dcff9640 1 -- 172.21.15.102:0/1987877079 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7fa3e0042de0 con 0x7fa3e8097d30 2024-09-13T17:43:44.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.629+0000 7fa3eeb9d640 1 -- 172.21.15.102:0/1987877079 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3e8117900 con 0x7fa3e8097d30 2024-09-13T17:43:44.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.634+0000 7fa3dcff9640 1 -- 172.21.15.102:0/1987877079 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fa3e0018070 con 0x7fa3e8097d30 2024-09-13T17:43:44.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.783+0000 7fa3eeb9d640 1 -- 172.21.15.102:0/1987877079 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa3e8064000 con 0x7fa3e8097d30 2024-09-13T17:43:44.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.784+0000 7fa3dcff9640 1 -- 172.21.15.102:0/1987877079 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa3e0024300 con 0x7fa3e8097d30 2024-09-13T17:43:44.786 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-09-13T17:43:44.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.786+0000 7fa3eeb9d640 1 -- 172.21.15.102:0/1987877079 >> v1:172.21.15.102:6816/34937 conn(0x7fa3d4031160 legacy=0x7fa3d4033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:44.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.786+0000 7fa3eeb9d640 1 -- 172.21.15.102:0/1987877079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3e8097d30 msgr2=0x7fa3e8117110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:44.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.786+0000 7fa3eeb9d640 1 --2- 172.21.15.102:0/1987877079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3e8097d30 0x7fa3e8117110 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009e50 tx=0x7fa3e000b040 comp rx=0 tx=0).stop 2024-09-13T17:43:44.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.786+0000 7fa3eeb9d640 1 -- 172.21.15.102:0/1987877079 shutdown_connections 2024-09-13T17:43:44.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.786+0000 7fa3eeb9d640 1 --2- 172.21.15.102:0/1987877079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3e8097d30 0x7fa3e8117110 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.786+0000 7fa3eeb9d640 1 --2- 172.21.15.102:0/1987877079 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa3e8097700 0x7fa3e8116bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.786+0000 7fa3eeb9d640 1 --2- 172.21.15.102:0/1987877079 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa3e8096df0 0x7fa3e811a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.786+0000 7fa3eeb9d640 1 -- 172.21.15.102:0/1987877079 >> 172.21.15.102:0/1987877079 conn(0x7fa3e809e720 msgr2=0x7fa3e809d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:44.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.786+0000 7fa3eeb9d640 1 -- 172.21.15.102:0/1987877079 shutdown_connections 2024-09-13T17:43:44.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.786+0000 7fa3eeb9d640 1 -- 172.21.15.102:0/1987877079 wait complete. 2024-09-13T17:43:44.798 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:43:44.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.891+0000 7f9fcfaaf640 1 Processor -- start 2024-09-13T17:43:44.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.891+0000 7f9fcfaaf640 1 -- start start 2024-09-13T17:43:44.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.892+0000 7f9fcfaaf640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9fc8132700 0x7f9fc8134ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:44.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.892+0000 7f9fcfaaf640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9fc8135020 0x7f9fc8137400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:44.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.892+0000 7f9fcfaaf640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fc80a2840 con 0x7f9fc812f8d0 2024-09-13T17:43:44.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.892+0000 7f9fcfaaf640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fc8096d80 con 0x7f9fc8135020 2024-09-13T17:43:44.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.892+0000 7f9fcfaaf640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc80a0e10 con 0x7f9fc8132700 2024-09-13T17:43:44.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.892+0000 7f9fcd04a640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9fc8132700 0x7f9fc8134ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:44.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.892+0000 7f9fcd84b640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f9fc812f8d0 0x7f9fc812fca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36350/0 (socket says 172.21.15.102:36350) 2024-09-13T17:43:44.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.892+0000 7f9fcd84b640 1 -- 172.21.15.102:0/3359646412 learned_addr learned my addr 172.21.15.102:0/3359646412 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:44.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.892+0000 7f9fce04c640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9fc8135020 0x7f9fc8137400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:44.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.892+0000 7f9fce04c640 1 -- 172.21.15.102:0/3359646412 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9fc8132700 msgr2=0x7f9fc8134ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:44.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.892+0000 7f9fce04c640 1 --2- 172.21.15.102:0/3359646412 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9fc8132700 0x7f9fc8134ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.892+0000 7f9fce04c640 1 -- 172.21.15.102:0/3359646412 >> v1:172.21.15.102:6789/0 conn(0x7f9fc812f8d0 legacy=0x7f9fc812fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:44.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.892+0000 7f9fce04c640 1 -- 172.21.15.102:0/3359646412 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc812f6b0 con 0x7f9fc8135020 2024-09-13T17:43:44.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.892+0000 7f9fcd04a640 1 --2- 172.21.15.102:0/3359646412 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9fc8132700 0x7f9fc8134ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:44.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.893+0000 7f9fce04c640 1 --2- 172.21.15.102:0/3359646412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fc8135020 0x7f9fc8137400 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0009850 tx=0x7f9fc00090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f13086ef8dd49a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:44.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.893+0000 7f9fcc849640 1 -- 172.21.15.102:0/3359646412 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc002ac80 con 0x7f9fc8135020 2024-09-13T17:43:44.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.893+0000 7f9fcc849640 1 -- 172.21.15.102:0/3359646412 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc002ade0 con 0x7f9fc8135020 2024-09-13T17:43:44.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.893+0000 7f9fcc849640 1 -- 172.21.15.102:0/3359646412 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0032670 con 0x7f9fc8135020 2024-09-13T17:43:44.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.894+0000 7f9fcfaaf640 1 -- 172.21.15.102:0/3359646412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fc8135020 msgr2=0x7f9fc8137400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:44.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.894+0000 7f9fcfaaf640 1 --2- 172.21.15.102:0/3359646412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fc8135020 0x7f9fc8137400 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0009850 tx=0x7f9fc00090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.894+0000 7f9fcfaaf640 1 -- 172.21.15.102:0/3359646412 shutdown_connections 2024-09-13T17:43:44.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.894+0000 7f9fcfaaf640 1 --2- 172.21.15.102:0/3359646412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fc8135020 0x7f9fc8137400 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.894+0000 7f9fcfaaf640 1 --2- 172.21.15.102:0/3359646412 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9fc8132700 0x7f9fc8134ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.894+0000 7f9fcfaaf640 1 -- 172.21.15.102:0/3359646412 >> 172.21.15.102:0/3359646412 conn(0x7f9fc80a3420 msgr2=0x7f9fc806e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:44.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.894+0000 7f9fcfaaf640 1 -- 172.21.15.102:0/3359646412 shutdown_connections 2024-09-13T17:43:44.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.894+0000 7f9fcfaaf640 1 -- 172.21.15.102:0/3359646412 wait complete. 2024-09-13T17:43:44.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.894+0000 7f9fcfaaf640 1 Processor -- start 2024-09-13T17:43:44.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.894+0000 7f9fcfaaf640 1 -- start start 2024-09-13T17:43:44.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.895+0000 7f9fcfaaf640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9fc812f8d0 0x7f9fc812ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:44.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.895+0000 7f9fcfaaf640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9fc8132700 0x7f9fc809a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:44.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.895+0000 7f9fcfaaf640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fc8135020 0x7f9fc809a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:44.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.895+0000 7f9fcfaaf640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc809bea0 con 0x7f9fc8135020 2024-09-13T17:43:44.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.895+0000 7f9fcfaaf640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc811a570 con 0x7f9fc8132700 2024-09-13T17:43:44.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.895+0000 7f9fcfaaf640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc8119500 con 0x7f9fc812f8d0 2024-09-13T17:43:44.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.895+0000 7f9fcd84b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9fc812f8d0 0x7f9fc812ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:44.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.895+0000 7f9fce04c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fc8135020 0x7f9fc809a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:44.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.895+0000 7f9fcd04a640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9fc8132700 0x7f9fc809a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:44.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.895+0000 7f9fce04c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fc8135020 0x7f9fc809a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57580/0 (socket says 172.21.15.102:57580) 2024-09-13T17:43:44.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.895+0000 7f9fcd84b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9fc812f8d0 0x7f9fc812ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:43260/0 (socket says 172.21.15.102:43260) 2024-09-13T17:43:44.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.895+0000 7f9fce04c640 1 -- 172.21.15.102:0/572449389 learned_addr learned my addr 172.21.15.102:0/572449389 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:44.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.895+0000 7f9fcd84b640 1 -- 172.21.15.102:0/572449389 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9fc8132700 msgr2=0x7f9fc809a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:44.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.895+0000 7f9fcd84b640 1 --2- 172.21.15.102:0/572449389 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9fc8132700 0x7f9fc809a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.895+0000 7f9fcd84b640 1 -- 172.21.15.102:0/572449389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fc8135020 msgr2=0x7f9fc809a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:44.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.895+0000 7f9fcd84b640 1 --2- 172.21.15.102:0/572449389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fc8135020 0x7f9fc809a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.895+0000 7f9fcd84b640 1 -- 172.21.15.102:0/572449389 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fbc009390 con 0x7f9fc812f8d0 2024-09-13T17:43:44.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.895+0000 7f9fce04c640 1 --2- 172.21.15.102:0/572449389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fc8135020 0x7f9fc809a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:44.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.895+0000 7f9fcd04a640 1 --2- 172.21.15.102:0/572449389 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9fc8132700 0x7f9fc809a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:44.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.895+0000 7f9fcd84b640 1 --2- 172.21.15.102:0/572449389 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9fc812f8d0 0x7f9fc812ca80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9fb800cc70 tx=0x7f9fb8001a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:44.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.896+0000 7f9fc5ffb640 1 -- 172.21.15.102:0/572449389 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb800ce30 con 0x7f9fc812f8d0 2024-09-13T17:43:44.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.896+0000 7f9fc5ffb640 1 -- 172.21.15.102:0/572449389 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb8004b20 con 0x7f9fc812f8d0 2024-09-13T17:43:44.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.896+0000 7f9fc5ffb640 1 -- 172.21.15.102:0/572449389 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb800a850 con 0x7f9fc812f8d0 2024-09-13T17:43:44.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.896+0000 7f9fcfaaf640 1 -- 172.21.15.102:0/572449389 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fc0009390 con 0x7f9fc812f8d0 2024-09-13T17:43:44.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.896+0000 7f9fcfaaf640 1 -- 172.21.15.102:0/572449389 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc8119f60 con 0x7f9fc812f8d0 2024-09-13T17:43:44.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.896+0000 7f9fc5ffb640 1 -- 172.21.15.102:0/572449389 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f9fb800ac80 con 0x7f9fc812f8d0 2024-09-13T17:43:44.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.897+0000 7f9fc5ffb640 1 -- 172.21.15.102:0/572449389 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f9fb8041900 con 0x7f9fc812f8d0 2024-09-13T17:43:44.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.897+0000 7f9fcfaaf640 1 -- 172.21.15.102:0/572449389 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fc80a18f0 con 0x7f9fc812f8d0 2024-09-13T17:43:44.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:44.901+0000 7f9fc5ffb640 1 -- 172.21.15.102:0/572449389 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f9fb8005030 con 0x7f9fc812f8d0 2024-09-13T17:43:45.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.052+0000 7f9fcfaaf640 1 -- 172.21.15.102:0/572449389 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9fc8064000 con 0x7f9fc812f8d0 2024-09-13T17:43:45.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.052+0000 7f9fc5ffb640 1 -- 172.21.15.102:0/572449389 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9fb801a310 con 0x7f9fc812f8d0 2024-09-13T17:43:45.054 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-09-13T17:43:45.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.054+0000 7f9fcfaaf640 1 -- 172.21.15.102:0/572449389 >> v1:172.21.15.102:6816/34937 conn(0x7f9fb4031190 legacy=0x7f9fb4033640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:45.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.054+0000 7f9fcfaaf640 1 -- 172.21.15.102:0/572449389 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9fc812f8d0 msgr2=0x7f9fc812ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:45.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.054+0000 7f9fcfaaf640 1 --2- 172.21.15.102:0/572449389 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9fc812f8d0 0x7f9fc812ca80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9fb800cc70 tx=0x7f9fb8001a10 comp rx=0 tx=0).stop 2024-09-13T17:43:45.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.054+0000 7f9fcfaaf640 1 -- 172.21.15.102:0/572449389 shutdown_connections 2024-09-13T17:43:45.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.054+0000 7f9fcfaaf640 1 --2- 172.21.15.102:0/572449389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fc8135020 0x7f9fc809a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.054+0000 7f9fcfaaf640 1 --2- 172.21.15.102:0/572449389 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9fc8132700 0x7f9fc809a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.054+0000 7f9fcfaaf640 1 --2- 172.21.15.102:0/572449389 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9fc812f8d0 0x7f9fc812ca80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.054+0000 7f9fcfaaf640 1 -- 172.21.15.102:0/572449389 >> 172.21.15.102:0/572449389 conn(0x7f9fc80a3420 msgr2=0x7f9fc8133e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:45.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.054+0000 7f9fcfaaf640 1 -- 172.21.15.102:0/572449389 shutdown_connections 2024-09-13T17:43:45.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.054+0000 7f9fcfaaf640 1 -- 172.21.15.102:0/572449389 wait complete. 2024-09-13T17:43:45.067 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:43:45.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.161+0000 7f2cea280640 1 Processor -- start 2024-09-13T17:43:45.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.161+0000 7f2cea280640 1 -- start start 2024-09-13T17:43:45.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.161+0000 7f2cea280640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2ce4096a70 0x7f2ce40987c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:45.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.161+0000 7f2cea280640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ce4098d00 0x7f2ce412c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:45.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.161+0000 7f2cea280640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ce4099110 0x7f2ce412c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:45.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.161+0000 7f2cea280640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ce4069c30 con 0x7f2ce4096a70 2024-09-13T17:43:45.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.161+0000 7f2cea280640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce406ed00 con 0x7f2ce4098d00 2024-09-13T17:43:45.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.161+0000 7f2cea280640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce40999e0 con 0x7f2ce4099110 2024-09-13T17:43:45.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.161+0000 7f2ce881d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ce4099110 0x7f2ce412c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:45.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.161+0000 7f2ce3fff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2ce4096a70 0x7f2ce40987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:45.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.161+0000 7f2ce37fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ce4098d00 0x7f2ce412c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:45.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.161+0000 7f2ce881d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ce4099110 0x7f2ce412c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:43268/0 (socket says 172.21.15.102:43268) 2024-09-13T17:43:45.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.161+0000 7f2ce3fff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2ce4096a70 0x7f2ce40987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57596/0 (socket says 172.21.15.102:57596) 2024-09-13T17:43:45.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.161+0000 7f2ce881d640 1 -- 172.21.15.102:0/178005078 learned_addr learned my addr 172.21.15.102:0/178005078 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:45.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.162+0000 7f2ce37fe640 1 -- 172.21.15.102:0/178005078 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ce4099110 msgr2=0x7f2ce412c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:45.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.162+0000 7f2ce37fe640 1 --2- 172.21.15.102:0/178005078 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ce4099110 0x7f2ce412c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.162+0000 7f2ce37fe640 1 -- 172.21.15.102:0/178005078 >> v2:172.21.15.102:3300/0 conn(0x7f2ce4096a70 msgr2=0x7f2ce40987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:45.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.162+0000 7f2ce37fe640 1 --2- 172.21.15.102:0/178005078 >> v2:172.21.15.102:3300/0 conn(0x7f2ce4096a70 0x7f2ce40987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.162+0000 7f2ce37fe640 1 -- 172.21.15.102:0/178005078 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce40997a0 con 0x7f2ce4098d00 2024-09-13T17:43:45.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.162+0000 7f2ce881d640 1 --2- 172.21.15.102:0/178005078 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ce4099110 0x7f2ce412c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:45.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.162+0000 7f2ce3fff640 1 --2- 172.21.15.102:0/178005078 >> v2:172.21.15.102:3300/0 conn(0x7f2ce4096a70 0x7f2ce40987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:45.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.162+0000 7f2ce37fe640 1 --2- 172.21.15.102:0/178005078 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ce4098d00 0x7f2ce412c2e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009850 tx=0x7f2cd80090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73b4115f16f26163 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:45.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.162+0000 7f2ce2ffd640 1 -- 172.21.15.102:0/178005078 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd802ce60 con 0x7f2ce4098d00 2024-09-13T17:43:45.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.162+0000 7f2ce2ffd640 1 -- 172.21.15.102:0/178005078 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd8034440 con 0x7f2ce4098d00 2024-09-13T17:43:45.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.162+0000 7f2ce2ffd640 1 -- 172.21.15.102:0/178005078 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd80347e0 con 0x7f2ce4098d00 2024-09-13T17:43:45.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.163+0000 7f2cea280640 1 -- 172.21.15.102:0/178005078 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ce4098d00 msgr2=0x7f2ce412c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:45.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.163+0000 7f2cea280640 1 --2- 172.21.15.102:0/178005078 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ce4098d00 0x7f2ce412c2e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009850 tx=0x7f2cd80090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.163+0000 7f2cea280640 1 -- 172.21.15.102:0/178005078 shutdown_connections 2024-09-13T17:43:45.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.163+0000 7f2cea280640 1 --2- 172.21.15.102:0/178005078 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ce4099110 0x7f2ce412c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.163+0000 7f2cea280640 1 --2- 172.21.15.102:0/178005078 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ce4098d00 0x7f2ce412c2e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.163+0000 7f2cea280640 1 --2- 172.21.15.102:0/178005078 >> v2:172.21.15.102:3300/0 conn(0x7f2ce4096a70 0x7f2ce40987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.163+0000 7f2cea280640 1 -- 172.21.15.102:0/178005078 >> 172.21.15.102:0/178005078 conn(0x7f2ce409e720 msgr2=0x7f2ce409eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:45.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.163+0000 7f2cea280640 1 -- 172.21.15.102:0/178005078 shutdown_connections 2024-09-13T17:43:45.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.163+0000 7f2cea280640 1 -- 172.21.15.102:0/178005078 wait complete. 2024-09-13T17:43:45.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.164+0000 7f2cea280640 1 Processor -- start 2024-09-13T17:43:45.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.164+0000 7f2cea280640 1 -- start start 2024-09-13T17:43:45.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.164+0000 7f2cea280640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ce4096a70 0x7f2ce4119c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:45.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.164+0000 7f2cea280640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ce4098d00 0x7f2ce411a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:45.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.164+0000 7f2cea280640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ce4099110 0x7f2ce411a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:45.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.164+0000 7f2cea280640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce409a410 con 0x7f2ce4096a70 2024-09-13T17:43:45.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.164+0000 7f2cea280640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce411d000 con 0x7f2ce4098d00 2024-09-13T17:43:45.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.164+0000 7f2cea280640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce411ca60 con 0x7f2ce4099110 2024-09-13T17:43:45.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.164+0000 7f2ce3fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ce4096a70 0x7f2ce4119c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:45.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.164+0000 7f2ce37fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ce4098d00 0x7f2ce411a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:45.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.164+0000 7f2ce3fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ce4096a70 0x7f2ce4119c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57604/0 (socket says 172.21.15.102:57604) 2024-09-13T17:43:45.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.164+0000 7f2ce881d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ce4099110 0x7f2ce411a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:45.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.164+0000 7f2ce3fff640 1 -- 172.21.15.102:0/3776325946 learned_addr learned my addr 172.21.15.102:0/3776325946 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:45.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.165+0000 7f2ce37fe640 1 -- 172.21.15.102:0/3776325946 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ce4099110 msgr2=0x7f2ce411a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:45.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.165+0000 7f2ce37fe640 1 --2- 172.21.15.102:0/3776325946 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ce4099110 0x7f2ce411a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.165+0000 7f2ce37fe640 1 -- 172.21.15.102:0/3776325946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ce4096a70 msgr2=0x7f2ce4119c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:45.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.165+0000 7f2ce37fe640 1 --2- 172.21.15.102:0/3776325946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ce4096a70 0x7f2ce4119c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.165+0000 7f2ce37fe640 1 -- 172.21.15.102:0/3776325946 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd4009390 con 0x7f2ce4098d00 2024-09-13T17:43:45.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.165+0000 7f2ce881d640 1 --2- 172.21.15.102:0/3776325946 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ce4099110 0x7f2ce411a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:45.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.165+0000 7f2ce3fff640 1 --2- 172.21.15.102:0/3776325946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ce4096a70 0x7f2ce4119c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:45.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.165+0000 7f2ce37fe640 1 --2- 172.21.15.102:0/3776325946 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ce4098d00 0x7f2ce411a170 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4099730 tx=0x7f2cd8034000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:45.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.165+0000 7f2ccbfff640 1 -- 172.21.15.102:0/3776325946 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8043070 con 0x7f2ce4098d00 2024-09-13T17:43:45.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.165+0000 7f2ccbfff640 1 -- 172.21.15.102:0/3776325946 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd8000c60 con 0x7f2ce4098d00 2024-09-13T17:43:45.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.165+0000 7f2ccbfff640 1 -- 172.21.15.102:0/3776325946 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8033cb0 con 0x7f2ce4098d00 2024-09-13T17:43:45.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.165+0000 7f2cea280640 1 -- 172.21.15.102:0/3776325946 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cdc009ae0 con 0x7f2ce4098d00 2024-09-13T17:43:45.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.165+0000 7f2cea280640 1 -- 172.21.15.102:0/3776325946 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd8009390 con 0x7f2ce4098d00 2024-09-13T17:43:45.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.166+0000 7f2ccbfff640 1 -- 172.21.15.102:0/3776325946 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f2cd803f690 con 0x7f2ce4098d00 2024-09-13T17:43:45.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.166+0000 7f2ccbfff640 1 -- 172.21.15.102:0/3776325946 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f2cd8062490 con 0x7f2ce4098d00 2024-09-13T17:43:45.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.166+0000 7f2cea280640 1 -- 172.21.15.102:0/3776325946 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce4064000 con 0x7f2ce4098d00 2024-09-13T17:43:45.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.170+0000 7f2ccbfff640 1 -- 172.21.15.102:0/3776325946 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f2cd8039070 con 0x7f2ce4098d00 2024-09-13T17:43:45.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.323+0000 7f2cea280640 1 -- 172.21.15.102:0/3776325946 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2ce411cc80 con 0x7f2ce4098d00 2024-09-13T17:43:45.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.323+0000 7f2ccbfff640 1 -- 172.21.15.102:0/3776325946 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2cd8033e10 con 0x7f2ce4098d00 2024-09-13T17:43:45.325 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-09-13T17:43:45.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.325+0000 7f2cea280640 1 -- 172.21.15.102:0/3776325946 >> v1:172.21.15.102:6816/34937 conn(0x7f2cd0031160 legacy=0x7f2cd0033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:45.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.325+0000 7f2cea280640 1 -- 172.21.15.102:0/3776325946 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ce4098d00 msgr2=0x7f2ce411a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:45.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.325+0000 7f2cea280640 1 --2- 172.21.15.102:0/3776325946 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ce4098d00 0x7f2ce411a170 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4099730 tx=0x7f2cd8034000 comp rx=0 tx=0).stop 2024-09-13T17:43:45.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.325+0000 7f2cea280640 1 -- 172.21.15.102:0/3776325946 shutdown_connections 2024-09-13T17:43:45.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.325+0000 7f2cea280640 1 --2- 172.21.15.102:0/3776325946 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ce4099110 0x7f2ce411a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.325+0000 7f2cea280640 1 --2- 172.21.15.102:0/3776325946 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ce4098d00 0x7f2ce411a170 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.325+0000 7f2cea280640 1 --2- 172.21.15.102:0/3776325946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ce4096a70 0x7f2ce4119c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.325+0000 7f2cea280640 1 -- 172.21.15.102:0/3776325946 >> 172.21.15.102:0/3776325946 conn(0x7f2ce409e720 msgr2=0x7f2ce409c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:45.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.325+0000 7f2cea280640 1 -- 172.21.15.102:0/3776325946 shutdown_connections 2024-09-13T17:43:45.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.325+0000 7f2cea280640 1 -- 172.21.15.102:0/3776325946 wait complete. 2024-09-13T17:43:45.338 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:43:45.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.429+0000 7f7e3ff5d640 1 Processor -- start 2024-09-13T17:43:45.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.429+0000 7f7e3ff5d640 1 -- start start 2024-09-13T17:43:45.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.429+0000 7f7e3ff5d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7e38128330 0x7f7e38128700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:45.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.429+0000 7f7e3ff5d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e3812bf50 0x7f7e38133b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:45.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.429+0000 7f7e3ff5d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e38097be0 con 0x7f7e38128c40 2024-09-13T17:43:45.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.429+0000 7f7e3ff5d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e38098e70 con 0x7f7e3812bf50 2024-09-13T17:43:45.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.429+0000 7f7e3ff5d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7e38098310 con 0x7f7e38128330 2024-09-13T17:43:45.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.429+0000 7f7e3dcf9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7e38128330 0x7f7e38128700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:45.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.429+0000 7f7e3e4fa640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e3812bf50 0x7f7e38133b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:45.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.429+0000 7f7e3d4f8640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f7e38128c40 0x7f7e3812b840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36352/0 (socket says 172.21.15.102:36352) 2024-09-13T17:43:45.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.430+0000 7f7e3d4f8640 1 -- 172.21.15.102:0/1782767813 learned_addr learned my addr 172.21.15.102:0/1782767813 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:45.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.430+0000 7f7e3ccf7640 1 -- 172.21.15.102:0/1782767813 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1026326392 0 0) 0x7f7e38097be0 con 0x7f7e38128c40 2024-09-13T17:43:45.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.430+0000 7f7e3e4fa640 1 -- 172.21.15.102:0/1782767813 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7e38128330 msgr2=0x7f7e38128700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:45.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.430+0000 7f7e3e4fa640 1 --2- 172.21.15.102:0/1782767813 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7e38128330 0x7f7e38128700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.430+0000 7f7e3e4fa640 1 -- 172.21.15.102:0/1782767813 >> v1:172.21.15.102:6789/0 conn(0x7f7e38128c40 legacy=0x7f7e3812b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:45.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.430+0000 7f7e3e4fa640 1 -- 172.21.15.102:0/1782767813 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e380985d0 con 0x7f7e3812bf50 2024-09-13T17:43:45.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.430+0000 7f7e3dcf9640 1 --2- 172.21.15.102:0/1782767813 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7e38128330 0x7f7e38128700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:45.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.430+0000 7f7e3e4fa640 1 --2- 172.21.15.102:0/1782767813 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e3812bf50 0x7f7e38133b90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7e30009e90 tx=0x7f7e30009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=455efd8c29353c39 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:45.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.431+0000 7f7e3ccf7640 1 -- 172.21.15.102:0/1782767813 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e30007550 con 0x7f7e3812bf50 2024-09-13T17:43:45.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.431+0000 7f7e3ccf7640 1 -- 172.21.15.102:0/1782767813 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e3002d050 con 0x7f7e3812bf50 2024-09-13T17:43:45.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.431+0000 7f7e3ccf7640 1 -- 172.21.15.102:0/1782767813 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e300078e0 con 0x7f7e3812bf50 2024-09-13T17:43:45.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.431+0000 7f7e3ff5d640 1 -- 172.21.15.102:0/1782767813 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e3812bf50 msgr2=0x7f7e38133b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:45.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.431+0000 7f7e3ff5d640 1 --2- 172.21.15.102:0/1782767813 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e3812bf50 0x7f7e38133b90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7e30009e90 tx=0x7f7e30009780 comp rx=0 tx=0).stop 2024-09-13T17:43:45.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.431+0000 7f7e3ff5d640 1 -- 172.21.15.102:0/1782767813 shutdown_connections 2024-09-13T17:43:45.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.431+0000 7f7e3ff5d640 1 --2- 172.21.15.102:0/1782767813 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e3812bf50 0x7f7e38133b90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.431+0000 7f7e3ff5d640 1 --2- 172.21.15.102:0/1782767813 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7e38128330 0x7f7e38128700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.431+0000 7f7e3ff5d640 1 -- 172.21.15.102:0/1782767813 >> 172.21.15.102:0/1782767813 conn(0x7f7e3809e820 msgr2=0x7f7e3809ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:45.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.431+0000 7f7e3ff5d640 1 -- 172.21.15.102:0/1782767813 shutdown_connections 2024-09-13T17:43:45.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.431+0000 7f7e3ff5d640 1 -- 172.21.15.102:0/1782767813 wait complete. 2024-09-13T17:43:45.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.432+0000 7f7e3ff5d640 1 Processor -- start 2024-09-13T17:43:45.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.432+0000 7f7e3ff5d640 1 -- start start 2024-09-13T17:43:45.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.432+0000 7f7e3ff5d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e38128330 0x7f7e38117a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:45.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.432+0000 7f7e3ff5d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7e38128c40 0x7f7e38114e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:45.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.432+0000 7f7e3ff5d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e3812bf50 0x7f7e381153c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:45.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.432+0000 7f7e3ff5d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3809a0a0 con 0x7f7e38128330 2024-09-13T17:43:45.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.432+0000 7f7e3ff5d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e38116fd0 con 0x7f7e3812bf50 2024-09-13T17:43:45.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.432+0000 7f7e3ff5d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7e38125a20 con 0x7f7e38128c40 2024-09-13T17:43:45.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.432+0000 7f7e3dcf9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e38128330 0x7f7e38117a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:45.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.433+0000 7f7e3dcf9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e38128330 0x7f7e38117a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57618/0 (socket says 172.21.15.102:57618) 2024-09-13T17:43:45.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.433+0000 7f7e3e4fa640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e3812bf50 0x7f7e381153c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:45.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.433+0000 7f7e3dcf9640 1 -- 172.21.15.102:0/904068972 learned_addr learned my addr 172.21.15.102:0/904068972 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:45.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.433+0000 7f7e3d4f8640 1 --2- 172.21.15.102:0/904068972 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7e38128c40 0x7f7e38114e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:45.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.433+0000 7f7e3dcf9640 1 -- 172.21.15.102:0/904068972 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7e38128c40 msgr2=0x7f7e38114e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:45.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.433+0000 7f7e3dcf9640 1 --2- 172.21.15.102:0/904068972 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7e38128c40 0x7f7e38114e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.433+0000 7f7e3dcf9640 1 -- 172.21.15.102:0/904068972 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e3812bf50 msgr2=0x7f7e381153c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:45.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.433+0000 7f7e3dcf9640 1 --2- 172.21.15.102:0/904068972 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e3812bf50 0x7f7e381153c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.433+0000 7f7e3dcf9640 1 -- 172.21.15.102:0/904068972 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3400a530 con 0x7f7e38128330 2024-09-13T17:43:45.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.433+0000 7f7e3e4fa640 1 --2- 172.21.15.102:0/904068972 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e3812bf50 0x7f7e381153c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:45.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.433+0000 7f7e3dcf9640 1 --2- 172.21.15.102:0/904068972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e38128330 0x7f7e38117a70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400ea10 tx=0x7f7e3400a2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:45.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.433+0000 7f7e2dffb640 1 -- 172.21.15.102:0/904068972 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3400ca40 con 0x7f7e38128330 2024-09-13T17:43:45.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.433+0000 7f7e2dffb640 1 -- 172.21.15.102:0/904068972 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e3400cba0 con 0x7f7e38128330 2024-09-13T17:43:45.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.433+0000 7f7e3ff5d640 1 -- 172.21.15.102:0/904068972 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e30009a20 con 0x7f7e38128330 2024-09-13T17:43:45.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.433+0000 7f7e2dffb640 1 -- 172.21.15.102:0/904068972 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3401a430 con 0x7f7e38128330 2024-09-13T17:43:45.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.433+0000 7f7e3ff5d640 1 -- 172.21.15.102:0/904068972 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e3806dea0 con 0x7f7e38128330 2024-09-13T17:43:45.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.434+0000 7f7e2dffb640 1 -- 172.21.15.102:0/904068972 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f7e340113f0 con 0x7f7e38128330 2024-09-13T17:43:45.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.434+0000 7f7e2dffb640 1 -- 172.21.15.102:0/904068972 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f7e34015030 con 0x7f7e38128330 2024-09-13T17:43:45.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.434+0000 7f7e3ff5d640 1 -- 172.21.15.102:0/904068972 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e38064000 con 0x7f7e38128330 2024-09-13T17:43:45.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.439+0000 7f7e2dffb640 1 -- 172.21.15.102:0/904068972 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f7e3401f080 con 0x7f7e38128330 2024-09-13T17:43:45.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.593+0000 7f7e3ff5d640 1 -- 172.21.15.102:0/904068972 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e38067690 con 0x7f7e38128330 2024-09-13T17:43:45.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.594+0000 7f7e2dffb640 1 -- 172.21.15.102:0/904068972 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+11660 (secure 0 0 0) 0x7f7e340193e0 con 0x7f7e38128330 2024-09-13T17:43:45.596 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:43:45.596 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":23,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:42:48.385896+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:04.672265+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:05.583134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.17","mappings":[{"from":3,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]},{"pgid":"4.25","mappings":[{"from":3,"to":2}]},{"pgid":"4.33","mappings":[{"from":3,"to":0}]},{"pgid":"4.3a","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:43:45.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.596+0000 7f7e3ff5d640 1 -- 172.21.15.102:0/904068972 >> v1:172.21.15.102:6816/34937 conn(0x7f7e1c031160 legacy=0x7f7e1c033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:45.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.596+0000 7f7e3ff5d640 1 -- 172.21.15.102:0/904068972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e38128330 msgr2=0x7f7e38117a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:45.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.596+0000 7f7e3ff5d640 1 --2- 172.21.15.102:0/904068972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e38128330 0x7f7e38117a70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400ea10 tx=0x7f7e3400a2b0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.596+0000 7f7e3ff5d640 1 -- 172.21.15.102:0/904068972 shutdown_connections 2024-09-13T17:43:45.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.596+0000 7f7e3ff5d640 1 --2- 172.21.15.102:0/904068972 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7e3812bf50 0x7f7e381153c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.596+0000 7f7e3ff5d640 1 --2- 172.21.15.102:0/904068972 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7e38128c40 0x7f7e38114e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.596+0000 7f7e3ff5d640 1 --2- 172.21.15.102:0/904068972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e38128330 0x7f7e38117a70 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.596+0000 7f7e3ff5d640 1 -- 172.21.15.102:0/904068972 >> 172.21.15.102:0/904068972 conn(0x7f7e3809e820 msgr2=0x7f7e38129740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:45.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.596+0000 7f7e3ff5d640 1 -- 172.21.15.102:0/904068972 shutdown_connections 2024-09-13T17:43:45.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.596+0000 7f7e3ff5d640 1 -- 172.21.15.102:0/904068972 wait complete. 2024-09-13T17:43:45.609 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:54.442612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T17:41:59.076782+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 16, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:42:04.672265+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:42:05.583134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:43:45.609 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:43:45.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.703+0000 7f059331d640 1 Processor -- start 2024-09-13T17:43:45.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.703+0000 7f059331d640 1 -- start start 2024-09-13T17:43:45.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.704+0000 7f059331d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f058c098010 0x7f058c130f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:45.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.704+0000 7f059331d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f058c0983e0 0x7f058c133470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:45.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.704+0000 7f059331d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f058c098c50 con 0x7f058c097530 2024-09-13T17:43:45.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.704+0000 7f059331d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f058c06ed00 con 0x7f058c0983e0 2024-09-13T17:43:45.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.704+0000 7f059331d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f058c068bd0 con 0x7f058c098010 2024-09-13T17:43:45.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.704+0000 7f05908b8640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f058c098010 0x7f058c130f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:45.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.704+0000 7f05910b9640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f058c097530 0x7f058c097900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36368/0 (socket says 172.21.15.102:36368) 2024-09-13T17:43:45.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.704+0000 7f05910b9640 1 -- 172.21.15.102:0/2409242474 learned_addr learned my addr 172.21.15.102:0/2409242474 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:45.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.704+0000 7f05918ba640 1 --2- 172.21.15.102:0/2409242474 >> v2:172.21.15.102:3300/0 conn(0x7f058c0983e0 0x7f058c133470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:45.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.704+0000 7f05918ba640 1 -- 172.21.15.102:0/2409242474 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f058c098010 msgr2=0x7f058c130f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:45.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.704+0000 7f05918ba640 1 --2- 172.21.15.102:0/2409242474 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f058c098010 0x7f058c130f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.704+0000 7f05918ba640 1 -- 172.21.15.102:0/2409242474 >> v1:172.21.15.102:6789/0 conn(0x7f058c097530 legacy=0x7f058c097900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:45.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.704+0000 7f0583fff640 1 -- 172.21.15.102:0/2409242474 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1803352772 0 0) 0x7f058c098c50 con 0x7f058c097530 2024-09-13T17:43:45.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.704+0000 7f05918ba640 1 -- 172.21.15.102:0/2409242474 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f058c099040 con 0x7f058c0983e0 2024-09-13T17:43:45.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.704+0000 7f05908b8640 1 --2- 172.21.15.102:0/2409242474 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f058c098010 0x7f058c130f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:45.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.705+0000 7f05918ba640 1 --2- 172.21.15.102:0/2409242474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f058c0983e0 0x7f058c133470 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0584009e60 tx=0x7f0584009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f66e007d060fea0b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:45.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.705+0000 7f0583fff640 1 -- 172.21.15.102:0/2409242474 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0584007550 con 0x7f058c0983e0 2024-09-13T17:43:45.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.705+0000 7f0583fff640 1 -- 172.21.15.102:0/2409242474 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058402d050 con 0x7f058c0983e0 2024-09-13T17:43:45.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.705+0000 7f0583fff640 1 -- 172.21.15.102:0/2409242474 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05840078e0 con 0x7f058c0983e0 2024-09-13T17:43:45.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.706+0000 7f059331d640 1 -- 172.21.15.102:0/2409242474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f058c0983e0 msgr2=0x7f058c133470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:45.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.706+0000 7f059331d640 1 --2- 172.21.15.102:0/2409242474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f058c0983e0 0x7f058c133470 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0584009e60 tx=0x7f0584009780 comp rx=0 tx=0).stop 2024-09-13T17:43:45.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.706+0000 7f059331d640 1 -- 172.21.15.102:0/2409242474 shutdown_connections 2024-09-13T17:43:45.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.706+0000 7f059331d640 1 --2- 172.21.15.102:0/2409242474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f058c0983e0 0x7f058c133470 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.706+0000 7f059331d640 1 --2- 172.21.15.102:0/2409242474 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f058c098010 0x7f058c130f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.706+0000 7f059331d640 1 -- 172.21.15.102:0/2409242474 >> 172.21.15.102:0/2409242474 conn(0x7f058c09e720 msgr2=0x7f058c09eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:45.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.706+0000 7f059331d640 1 -- 172.21.15.102:0/2409242474 shutdown_connections 2024-09-13T17:43:45.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.706+0000 7f059331d640 1 -- 172.21.15.102:0/2409242474 wait complete. 2024-09-13T17:43:45.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.706+0000 7f059331d640 1 Processor -- start 2024-09-13T17:43:45.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.706+0000 7f059331d640 1 -- start start 2024-09-13T17:43:45.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.707+0000 7f059331d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f058c097530 0x7f058c13cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:45.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.707+0000 7f059331d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f058c098010 0x7f058c13cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:45.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.707+0000 7f059331d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f058c0983e0 0x7f058c1171f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:45.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.707+0000 7f059331d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058c11a950 con 0x7f058c097530 2024-09-13T17:43:45.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.707+0000 7f059331d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f058c119260 con 0x7f058c0983e0 2024-09-13T17:43:45.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.707+0000 7f059331d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f058c134830 con 0x7f058c098010 2024-09-13T17:43:45.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.707+0000 7f05910b9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f058c097530 0x7f058c13cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:45.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.707+0000 7f05908b8640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f058c098010 0x7f058c13cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:45.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.707+0000 7f05910b9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f058c097530 0x7f058c13cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57640/0 (socket says 172.21.15.102:57640) 2024-09-13T17:43:45.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.707+0000 7f05908b8640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f058c098010 0x7f058c13cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:43314/0 (socket says 172.21.15.102:43314) 2024-09-13T17:43:45.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.707+0000 7f05910b9640 1 -- 172.21.15.102:0/3175374836 learned_addr learned my addr 172.21.15.102:0/3175374836 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:45.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.707+0000 7f05918ba640 1 --2- 172.21.15.102:0/3175374836 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f058c0983e0 0x7f058c1171f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:45.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.707+0000 7f05910b9640 1 -- 172.21.15.102:0/3175374836 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f058c098010 msgr2=0x7f058c13cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:45.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.707+0000 7f05910b9640 1 --2- 172.21.15.102:0/3175374836 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f058c098010 0x7f058c13cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.707+0000 7f05910b9640 1 -- 172.21.15.102:0/3175374836 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f058c0983e0 msgr2=0x7f058c1171f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:45.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.707+0000 7f05910b9640 1 --2- 172.21.15.102:0/3175374836 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f058c0983e0 0x7f058c1171f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.707+0000 7f05910b9640 1 -- 172.21.15.102:0/3175374836 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f057c009390 con 0x7f058c097530 2024-09-13T17:43:45.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.707+0000 7f05908b8640 1 --2- 172.21.15.102:0/3175374836 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f058c098010 0x7f058c13cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:45.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.707+0000 7f05918ba640 1 --2- 172.21.15.102:0/3175374836 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f058c0983e0 0x7f058c1171f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:45.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.708+0000 7f05910b9640 1 --2- 172.21.15.102:0/3175374836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f058c097530 0x7f058c13cab0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f058800c5e0 tx=0x7f0588007d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:45.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.708+0000 7f05817fa640 1 -- 172.21.15.102:0/3175374836 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0588005aa0 con 0x7f058c097530 2024-09-13T17:43:45.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.708+0000 7f05817fa640 1 -- 172.21.15.102:0/3175374836 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0588005c20 con 0x7f058c097530 2024-09-13T17:43:45.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.708+0000 7f059331d640 1 -- 172.21.15.102:0/3175374836 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0584009a20 con 0x7f058c097530 2024-09-13T17:43:45.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.708+0000 7f05817fa640 1 -- 172.21.15.102:0/3175374836 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0588007340 con 0x7f058c097530 2024-09-13T17:43:45.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.708+0000 7f059331d640 1 -- 172.21.15.102:0/3175374836 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f058c118d20 con 0x7f058c097530 2024-09-13T17:43:45.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.709+0000 7f05817fa640 1 -- 172.21.15.102:0/3175374836 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f058801b400 con 0x7f058c097530 2024-09-13T17:43:45.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.709+0000 7f05817fa640 1 -- 172.21.15.102:0/3175374836 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f0588041ae0 con 0x7f058c097530 2024-09-13T17:43:45.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.709+0000 7f059331d640 1 -- 172.21.15.102:0/3175374836 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f058c06cc40 con 0x7f058c097530 2024-09-13T17:43:45.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.713+0000 7f05817fa640 1 -- 172.21.15.102:0/3175374836 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f058801be60 con 0x7f058c097530 2024-09-13T17:43:45.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.865+0000 7f059331d640 1 -- 172.21.15.102:0/3175374836 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f058c11aa90 con 0x7f058c097530 2024-09-13T17:43:45.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.865+0000 7f05817fa640 1 -- 172.21.15.102:0/3175374836 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f058c11aa90 con 0x7f058c097530 2024-09-13T17:43:45.867 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-09-13T17:43:45.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.867+0000 7f059331d640 1 -- 172.21.15.102:0/3175374836 >> v1:172.21.15.102:6816/34937 conn(0x7f0570031160 legacy=0x7f0570033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:45.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.867+0000 7f059331d640 1 -- 172.21.15.102:0/3175374836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f058c097530 msgr2=0x7f058c13cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:45.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.867+0000 7f059331d640 1 --2- 172.21.15.102:0/3175374836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f058c097530 0x7f058c13cab0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f058800c5e0 tx=0x7f0588007d00 comp rx=0 tx=0).stop 2024-09-13T17:43:45.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.867+0000 7f059331d640 1 -- 172.21.15.102:0/3175374836 shutdown_connections 2024-09-13T17:43:45.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.867+0000 7f059331d640 1 --2- 172.21.15.102:0/3175374836 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f058c0983e0 0x7f058c1171f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.867+0000 7f059331d640 1 --2- 172.21.15.102:0/3175374836 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f058c098010 0x7f058c13cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.867+0000 7f059331d640 1 --2- 172.21.15.102:0/3175374836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f058c097530 0x7f058c13cab0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.867+0000 7f059331d640 1 -- 172.21.15.102:0/3175374836 >> 172.21.15.102:0/3175374836 conn(0x7f058c09e720 msgr2=0x7f058c09c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:45.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.867+0000 7f059331d640 1 -- 172.21.15.102:0/3175374836 shutdown_connections 2024-09-13T17:43:45.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.867+0000 7f059331d640 1 -- 172.21.15.102:0/3175374836 wait complete. 2024-09-13T17:43:45.879 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-13T17:43:45.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.973+0000 7f65ea8c0640 1 Processor -- start 2024-09-13T17:43:45.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.973+0000 7f65ea8c0640 1 -- start start 2024-09-13T17:43:45.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.974+0000 7f65ea8c0640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f65e4096df0 0x7f65e40971c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:45.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.974+0000 7f65ea8c0640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65e4097700 0x7f65e412d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:45.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.974+0000 7f65ea8c0640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e406a850 con 0x7f65e4097d30 2024-09-13T17:43:45.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.974+0000 7f65ea8c0640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e4098290 con 0x7f65e4096df0 2024-09-13T17:43:45.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.974+0000 7f65ea8c0640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f65e4096bf0 con 0x7f65e4097700 2024-09-13T17:43:45.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.974+0000 7f65e3fff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f65e4096df0 0x7f65e40971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:45.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.974+0000 7f65e8e5d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65e4097700 0x7f65e412d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:45.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.974+0000 7f65e3fff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f65e4096df0 0x7f65e40971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57654/0 (socket says 172.21.15.102:57654) 2024-09-13T17:43:45.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.974+0000 7f65e3fff640 1 -- 172.21.15.102:0/3416397736 learned_addr learned my addr 172.21.15.102:0/3416397736 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:45.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.974+0000 7f65e3fff640 1 -- 172.21.15.102:0/3416397736 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65e4097700 msgr2=0x7f65e412d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:45.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.974+0000 7f65e3fff640 1 --2- 172.21.15.102:0/3416397736 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65e4097700 0x7f65e412d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.974+0000 7f65e3fff640 1 -- 172.21.15.102:0/3416397736 >> v1:172.21.15.102:6789/0 conn(0x7f65e4097d30 legacy=0x7f65e412d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:45.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.974+0000 7f65e3fff640 1 -- 172.21.15.102:0/3416397736 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e406a590 con 0x7f65e4096df0 2024-09-13T17:43:45.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.974+0000 7f65e37fe640 1 -- 172.21.15.102:0/3416397736 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1183988260 0 0) 0x7f65e406a850 con 0x7f65e4097d30 2024-09-13T17:43:45.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.974+0000 7f65e8e5d640 1 --2- 172.21.15.102:0/3416397736 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65e4097700 0x7f65e412d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:45.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.975+0000 7f65e3fff640 1 --2- 172.21.15.102:0/3416397736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65e4096df0 0x7f65e40971c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f65d4009940 tx=0x7f65d40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890194d1e99ed2c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:45.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.975+0000 7f65e37fe640 1 -- 172.21.15.102:0/3416397736 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d402adb0 con 0x7f65e4096df0 2024-09-13T17:43:45.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.975+0000 7f65e37fe640 1 -- 172.21.15.102:0/3416397736 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d4032440 con 0x7f65e4096df0 2024-09-13T17:43:45.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.975+0000 7f65e37fe640 1 -- 172.21.15.102:0/3416397736 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d40327e0 con 0x7f65e4096df0 2024-09-13T17:43:45.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.976+0000 7f65ea8c0640 1 -- 172.21.15.102:0/3416397736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65e4096df0 msgr2=0x7f65e40971c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:45.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.976+0000 7f65ea8c0640 1 --2- 172.21.15.102:0/3416397736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65e4096df0 0x7f65e40971c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f65d4009940 tx=0x7f65d40091d0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.976+0000 7f65ea8c0640 1 -- 172.21.15.102:0/3416397736 shutdown_connections 2024-09-13T17:43:45.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.976+0000 7f65ea8c0640 1 --2- 172.21.15.102:0/3416397736 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65e4097700 0x7f65e412d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.976+0000 7f65ea8c0640 1 --2- 172.21.15.102:0/3416397736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65e4096df0 0x7f65e40971c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.976+0000 7f65ea8c0640 1 -- 172.21.15.102:0/3416397736 >> 172.21.15.102:0/3416397736 conn(0x7f65e409e720 msgr2=0x7f65e409eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:45.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.976+0000 7f65ea8c0640 1 -- 172.21.15.102:0/3416397736 shutdown_connections 2024-09-13T17:43:45.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.976+0000 7f65ea8c0640 1 -- 172.21.15.102:0/3416397736 wait complete. 2024-09-13T17:43:45.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.976+0000 7f65ea8c0640 1 Processor -- start 2024-09-13T17:43:45.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.977+0000 7f65ea8c0640 1 -- start start 2024-09-13T17:43:45.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.977+0000 7f65ea8c0640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65e4096df0 0x7f65e411f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:45.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.977+0000 7f65ea8c0640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65e4097700 0x7f65e4098a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:45.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.977+0000 7f65ea8c0640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65e4097d30 0x7f65e4098f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:45.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.977+0000 7f65ea8c0640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e4069df0 con 0x7f65e4097700 2024-09-13T17:43:45.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.977+0000 7f65ea8c0640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e409a040 con 0x7f65e4097d30 2024-09-13T17:43:45.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.977+0000 7f65ea8c0640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f65e411ecb0 con 0x7f65e4096df0 2024-09-13T17:43:45.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.977+0000 7f65e3fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65e4096df0 0x7f65e411f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:45.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.977+0000 7f65dbfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65e4097700 0x7f65e4098a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:45.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.977+0000 7f65e3fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65e4096df0 0x7f65e411f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:43330/0 (socket says 172.21.15.102:43330) 2024-09-13T17:43:45.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.977+0000 7f65e3fff640 1 -- 172.21.15.102:0/1599882894 learned_addr learned my addr 172.21.15.102:0/1599882894 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:45.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.977+0000 7f65e8e5d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65e4097d30 0x7f65e4098f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:45.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.977+0000 7f65e3fff640 1 -- 172.21.15.102:0/1599882894 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65e4097d30 msgr2=0x7f65e4098f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:45.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.977+0000 7f65e3fff640 1 --2- 172.21.15.102:0/1599882894 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65e4097d30 0x7f65e4098f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.977+0000 7f65e3fff640 1 -- 172.21.15.102:0/1599882894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65e4097700 msgr2=0x7f65e4098a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:45.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.977+0000 7f65e3fff640 1 --2- 172.21.15.102:0/1599882894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65e4097700 0x7f65e4098a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:45.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.977+0000 7f65e3fff640 1 -- 172.21.15.102:0/1599882894 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65dc009a20 con 0x7f65e4096df0 2024-09-13T17:43:45.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.977+0000 7f65dbfff640 1 --2- 172.21.15.102:0/1599882894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65e4097700 0x7f65e4098a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:45.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.977+0000 7f65e8e5d640 1 --2- 172.21.15.102:0/1599882894 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65e4097d30 0x7f65e4098f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:45.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.978+0000 7f65e3fff640 1 --2- 172.21.15.102:0/1599882894 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65e4096df0 0x7f65e411f420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f65d402a7f0 tx=0x7f65d4032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:45.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.978+0000 7f65e0ff9640 1 -- 172.21.15.102:0/1599882894 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d403c6f0 con 0x7f65e4096df0 2024-09-13T17:43:45.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.978+0000 7f65e0ff9640 1 -- 172.21.15.102:0/1599882894 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d403c870 con 0x7f65e4096df0 2024-09-13T17:43:45.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.978+0000 7f65ea8c0640 1 -- 172.21.15.102:0/1599882894 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d4009450 con 0x7f65e4096df0 2024-09-13T17:43:45.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.978+0000 7f65e0ff9640 1 -- 172.21.15.102:0/1599882894 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d403cc30 con 0x7f65e4096df0 2024-09-13T17:43:45.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.978+0000 7f65ea8c0640 1 -- 172.21.15.102:0/1599882894 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e40994b0 con 0x7f65e4096df0 2024-09-13T17:43:45.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.978+0000 7f65e0ff9640 1 -- 172.21.15.102:0/1599882894 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f65d4039050 con 0x7f65e4096df0 2024-09-13T17:43:45.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.979+0000 7f65e0ff9640 1 -- 172.21.15.102:0/1599882894 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f65d4062390 con 0x7f65e4096df0 2024-09-13T17:43:45.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.979+0000 7f65ea8c0640 1 -- 172.21.15.102:0/1599882894 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65e411e950 con 0x7f65e4096df0 2024-09-13T17:43:45.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:45.983+0000 7f65e0ff9640 1 -- 172.21.15.102:0/1599882894 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f65e411e950 con 0x7f65e4096df0 2024-09-13T17:43:46.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.135+0000 7f65ea8c0640 1 -- 172.21.15.102:0/1599882894 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f65e411e950 con 0x7f65e4096df0 2024-09-13T17:43:46.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.135+0000 7f65e0ff9640 1 -- 172.21.15.102:0/1599882894 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f65e411e950 con 0x7f65e4096df0 2024-09-13T17:43:46.138 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-09-13T17:43:46.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.137+0000 7f65ea8c0640 1 -- 172.21.15.102:0/1599882894 >> v1:172.21.15.102:6816/34937 conn(0x7f65c4031160 legacy=0x7f65c4033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:46.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.137+0000 7f65ea8c0640 1 -- 172.21.15.102:0/1599882894 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65e4096df0 msgr2=0x7f65e411f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:46.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.137+0000 7f65ea8c0640 1 --2- 172.21.15.102:0/1599882894 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65e4096df0 0x7f65e411f420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f65d402a7f0 tx=0x7f65d4032000 comp rx=0 tx=0).stop 2024-09-13T17:43:46.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.137+0000 7f65ea8c0640 1 -- 172.21.15.102:0/1599882894 shutdown_connections 2024-09-13T17:43:46.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.137+0000 7f65ea8c0640 1 --2- 172.21.15.102:0/1599882894 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f65e4097d30 0x7f65e4098f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:46.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.137+0000 7f65ea8c0640 1 --2- 172.21.15.102:0/1599882894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f65e4097700 0x7f65e4098a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:46.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.137+0000 7f65ea8c0640 1 --2- 172.21.15.102:0/1599882894 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f65e4096df0 0x7f65e411f420 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:46.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.137+0000 7f65ea8c0640 1 -- 172.21.15.102:0/1599882894 >> 172.21.15.102:0/1599882894 conn(0x7f65e409e720 msgr2=0x7f65e412d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:46.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.137+0000 7f65ea8c0640 1 -- 172.21.15.102:0/1599882894 shutdown_connections 2024-09-13T17:43:46.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.138+0000 7f65ea8c0640 1 -- 172.21.15.102:0/1599882894 wait complete. 2024-09-13T17:43:46.151 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:43:46.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.245+0000 7ffa61e60640 1 Processor -- start 2024-09-13T17:43:46.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.245+0000 7ffa61e60640 1 -- start start 2024-09-13T17:43:46.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.245+0000 7ffa61e60640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa5c097340 0x7ffa5c132510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:46.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.245+0000 7ffa61e60640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ffa5c097710 0x7ffa5c134a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:46.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.245+0000 7ffa61e60640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa5c098040 con 0x7ffa5c09a620 2024-09-13T17:43:46.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.245+0000 7ffa61e60640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa5c09a9f0 con 0x7ffa5c097710 2024-09-13T17:43:46.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.245+0000 7ffa61e60640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c0982a0 con 0x7ffa5c097340 2024-09-13T17:43:46.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.245+0000 7ffa5affd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa5c097340 0x7ffa5c132510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:46.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.246+0000 7ffa5bfff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ffa5c097710 0x7ffa5c134a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:46.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.246+0000 7ffa5affd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa5c097340 0x7ffa5c132510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:43332/0 (socket says 172.21.15.102:43332) 2024-09-13T17:43:46.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.246+0000 7ffa5affd640 1 -- 172.21.15.102:0/1821028433 learned_addr learned my addr 172.21.15.102:0/1821028433 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:46.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.246+0000 7ffa5affd640 1 -- 172.21.15.102:0/1821028433 >> v2:172.21.15.102:3300/0 conn(0x7ffa5c097710 msgr2=0x7ffa5c134a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:46.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.246+0000 7ffa5affd640 1 --2- 172.21.15.102:0/1821028433 >> v2:172.21.15.102:3300/0 conn(0x7ffa5c097710 0x7ffa5c134a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:46.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.246+0000 7ffa5affd640 1 -- 172.21.15.102:0/1821028433 >> v1:172.21.15.102:6789/0 conn(0x7ffa5c09a620 legacy=0x7ffa5c096c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:46.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.246+0000 7ffa5affd640 1 -- 172.21.15.102:0/1821028433 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa5c09ad50 con 0x7ffa5c097340 2024-09-13T17:43:46.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.246+0000 7ffa5affd640 1 --2- 172.21.15.102:0/1821028433 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa5c097340 0x7ffa5c132510 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ffa50009910 tx=0x7ffa500091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53eea3e0e0d8debb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:46.253 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.247+0000 7ffa5a7fc640 1 -- 172.21.15.102:0/1821028433 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5002adb0 con 0x7ffa5c097340 2024-09-13T17:43:46.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.247+0000 7ffa5a7fc640 1 -- 172.21.15.102:0/1821028433 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa50032440 con 0x7ffa5c097340 2024-09-13T17:43:46.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.247+0000 7ffa5a7fc640 1 -- 172.21.15.102:0/1821028433 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa50032820 con 0x7ffa5c097340 2024-09-13T17:43:46.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.247+0000 7ffa61e60640 1 -- 172.21.15.102:0/1821028433 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa5c097340 msgr2=0x7ffa5c132510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:46.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.247+0000 7ffa61e60640 1 --2- 172.21.15.102:0/1821028433 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa5c097340 0x7ffa5c132510 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ffa50009910 tx=0x7ffa500091d0 comp rx=0 tx=0).stop 2024-09-13T17:43:46.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.247+0000 7ffa61e60640 1 -- 172.21.15.102:0/1821028433 shutdown_connections 2024-09-13T17:43:46.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.247+0000 7ffa61e60640 1 --2- 172.21.15.102:0/1821028433 >> v2:172.21.15.102:3300/0 conn(0x7ffa5c097710 0x7ffa5c134a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:46.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.247+0000 7ffa61e60640 1 --2- 172.21.15.102:0/1821028433 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa5c097340 0x7ffa5c132510 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:46.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.247+0000 7ffa61e60640 1 -- 172.21.15.102:0/1821028433 >> 172.21.15.102:0/1821028433 conn(0x7ffa5c09e6b0 msgr2=0x7ffa5c09eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:46.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.247+0000 7ffa61e60640 1 -- 172.21.15.102:0/1821028433 shutdown_connections 2024-09-13T17:43:46.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.247+0000 7ffa61e60640 1 -- 172.21.15.102:0/1821028433 wait complete. 2024-09-13T17:43:46.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.248+0000 7ffa61e60640 1 Processor -- start 2024-09-13T17:43:46.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.248+0000 7ffa61e60640 1 -- start start 2024-09-13T17:43:46.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.249+0000 7ffa61e60640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa5c097340 0x7ffa5c115d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:46.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.249+0000 7ffa61e60640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffa5c097710 0x7ffa5c116240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:46.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.249+0000 7ffa61e60640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa5c09a620 0x7ffa5c116780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:46.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.249+0000 7ffa61e60640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c069d20 con 0x7ffa5c09a620 2024-09-13T17:43:46.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.249+0000 7ffa61e60640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c117170 con 0x7ffa5c097710 2024-09-13T17:43:46.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.249+0000 7ffa61e60640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c119720 con 0x7ffa5c097340 2024-09-13T17:43:46.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.249+0000 7ffa5b7fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa5c097340 0x7ffa5c115d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:46.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.249+0000 7ffa5bfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa5c09a620 0x7ffa5c116780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:46.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.249+0000 7ffa5affd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffa5c097710 0x7ffa5c116240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:46.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.249+0000 7ffa5bfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa5c09a620 0x7ffa5c116780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57674/0 (socket says 172.21.15.102:57674) 2024-09-13T17:43:46.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.249+0000 7ffa5b7fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa5c097340 0x7ffa5c115d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:43342/0 (socket says 172.21.15.102:43342) 2024-09-13T17:43:46.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.249+0000 7ffa5bfff640 1 -- 172.21.15.102:0/664967336 learned_addr learned my addr 172.21.15.102:0/664967336 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:46.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.249+0000 7ffa5bfff640 1 -- 172.21.15.102:0/664967336 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa5c097340 msgr2=0x7ffa5c115d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:46.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.249+0000 7ffa5bfff640 1 --2- 172.21.15.102:0/664967336 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa5c097340 0x7ffa5c115d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:46.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.249+0000 7ffa5bfff640 1 -- 172.21.15.102:0/664967336 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffa5c097710 msgr2=0x7ffa5c116240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:46.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.249+0000 7ffa5bfff640 1 --2- 172.21.15.102:0/664967336 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffa5c097710 0x7ffa5c116240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:46.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.249+0000 7ffa5bfff640 1 -- 172.21.15.102:0/664967336 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa50009450 con 0x7ffa5c09a620 2024-09-13T17:43:46.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.249+0000 7ffa5bfff640 1 --2- 172.21.15.102:0/664967336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa5c09a620 0x7ffa5c116780 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009e90 tx=0x7ffa5400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:46.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.249+0000 7ffa437fe640 1 -- 172.21.15.102:0/664967336 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5400ce10 con 0x7ffa5c09a620 2024-09-13T17:43:46.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.249+0000 7ffa437fe640 1 -- 172.21.15.102:0/664967336 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5400f040 con 0x7ffa5c09a620 2024-09-13T17:43:46.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.249+0000 7ffa437fe640 1 -- 172.21.15.102:0/664967336 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa540146d0 con 0x7ffa5c09a620 2024-09-13T17:43:46.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.249+0000 7ffa61e60640 1 -- 172.21.15.102:0/664967336 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa5c134a70 con 0x7ffa5c09a620 2024-09-13T17:43:46.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.249+0000 7ffa61e60640 1 -- 172.21.15.102:0/664967336 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa5c117880 con 0x7ffa5c09a620 2024-09-13T17:43:46.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.250+0000 7ffa437fe640 1 -- 172.21.15.102:0/664967336 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7ffa54014a80 con 0x7ffa5c09a620 2024-09-13T17:43:46.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.250+0000 7ffa437fe640 1 -- 172.21.15.102:0/664967336 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7ffa54042c50 con 0x7ffa5c09a620 2024-09-13T17:43:46.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.250+0000 7ffa61e60640 1 -- 172.21.15.102:0/664967336 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa5c119ce0 con 0x7ffa5c09a620 2024-09-13T17:43:46.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.255+0000 7ffa437fe640 1 -- 172.21.15.102:0/664967336 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7ffa54018070 con 0x7ffa5c09a620 2024-09-13T17:43:46.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.405+0000 7ffa61e60640 1 -- 172.21.15.102:0/664967336 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffa5c064000 con 0x7ffa5c09a620 2024-09-13T17:43:46.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.405+0000 7ffa437fe640 1 -- 172.21.15.102:0/664967336 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffa540118d0 con 0x7ffa5c09a620 2024-09-13T17:43:46.407 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-09-13T17:43:46.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.407+0000 7ffa61e60640 1 -- 172.21.15.102:0/664967336 >> v1:172.21.15.102:6816/34937 conn(0x7ffa48031160 legacy=0x7ffa48033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:46.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.407+0000 7ffa61e60640 1 -- 172.21.15.102:0/664967336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa5c09a620 msgr2=0x7ffa5c116780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:46.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.407+0000 7ffa61e60640 1 --2- 172.21.15.102:0/664967336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa5c09a620 0x7ffa5c116780 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009e90 tx=0x7ffa5400b040 comp rx=0 tx=0).stop 2024-09-13T17:43:46.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.407+0000 7ffa61e60640 1 -- 172.21.15.102:0/664967336 shutdown_connections 2024-09-13T17:43:46.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.407+0000 7ffa61e60640 1 --2- 172.21.15.102:0/664967336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa5c09a620 0x7ffa5c116780 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:46.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.407+0000 7ffa61e60640 1 --2- 172.21.15.102:0/664967336 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffa5c097710 0x7ffa5c116240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:46.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.407+0000 7ffa61e60640 1 --2- 172.21.15.102:0/664967336 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa5c097340 0x7ffa5c115d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:46.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.407+0000 7ffa61e60640 1 -- 172.21.15.102:0/664967336 >> 172.21.15.102:0/664967336 conn(0x7ffa5c09e6b0 msgr2=0x7ffa5c09d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:46.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.407+0000 7ffa61e60640 1 -- 172.21.15.102:0/664967336 shutdown_connections 2024-09-13T17:43:46.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.407+0000 7ffa61e60640 1 -- 172.21.15.102:0/664967336 wait complete. 2024-09-13T17:43:46.420 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:43:46.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.514+0000 7f595e51d640 1 Processor -- start 2024-09-13T17:43:46.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.514+0000 7f595e51d640 1 -- start start 2024-09-13T17:43:46.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.514+0000 7f595e51d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5958096a70 0x7f59580987c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:46.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.514+0000 7f595e51d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5958098d00 0x7f595812c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:46.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.514+0000 7f595e51d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5958099110 0x7f595812c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:46.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.514+0000 7f595e51d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5958069c30 con 0x7f5958098d00 2024-09-13T17:43:46.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.514+0000 7f595e51d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595806ed00 con 0x7f5958099110 2024-09-13T17:43:46.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.514+0000 7f595e51d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f59580999e0 con 0x7f5958096a70 2024-09-13T17:43:46.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.514+0000 7f5957fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5958096a70 0x7f59580987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:46.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.514+0000 7f59577fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5958098d00 0x7f595812c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:46.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.514+0000 7f595caba640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5958099110 0x7f595812c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:46.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.514+0000 7f5957fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5958096a70 0x7f59580987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:43354/0 (socket says 172.21.15.102:43354) 2024-09-13T17:43:46.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.514+0000 7f59577fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5958098d00 0x7f595812c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57676/0 (socket says 172.21.15.102:57676) 2024-09-13T17:43:46.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.514+0000 7f5957fff640 1 -- 172.21.15.102:0/1117591245 learned_addr learned my addr 172.21.15.102:0/1117591245 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:46.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.515+0000 7f595caba640 1 -- 172.21.15.102:0/1117591245 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5958096a70 msgr2=0x7f59580987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:46.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.515+0000 7f595caba640 1 --2- 172.21.15.102:0/1117591245 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5958096a70 0x7f59580987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:46.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.515+0000 7f595caba640 1 -- 172.21.15.102:0/1117591245 >> v2:172.21.15.102:3300/0 conn(0x7f5958098d00 msgr2=0x7f595812c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:46.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.515+0000 7f595caba640 1 --2- 172.21.15.102:0/1117591245 >> v2:172.21.15.102:3300/0 conn(0x7f5958098d00 0x7f595812c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:46.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.515+0000 7f595caba640 1 -- 172.21.15.102:0/1117591245 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59580997a0 con 0x7f5958099110 2024-09-13T17:43:46.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.515+0000 7f5957fff640 1 --2- 172.21.15.102:0/1117591245 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5958096a70 0x7f59580987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:46.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.515+0000 7f59577fe640 1 --2- 172.21.15.102:0/1117591245 >> v2:172.21.15.102:3300/0 conn(0x7f5958098d00 0x7f595812c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:46.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.515+0000 7f595caba640 1 --2- 172.21.15.102:0/1117591245 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5958099110 0x7f595812c820 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f595000d0d0 tx=0x7f595000b990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ae8363a9ae03b4f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:46.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.515+0000 7f5956ffd640 1 -- 172.21.15.102:0/1117591245 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5950030050 con 0x7f5958099110 2024-09-13T17:43:46.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.515+0000 7f5956ffd640 1 -- 172.21.15.102:0/1117591245 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595000b2e0 con 0x7f5958099110 2024-09-13T17:43:46.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.515+0000 7f5956ffd640 1 -- 172.21.15.102:0/1117591245 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5950009780 con 0x7f5958099110 2024-09-13T17:43:46.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.516+0000 7f595e51d640 1 -- 172.21.15.102:0/1117591245 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5958099110 msgr2=0x7f595812c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:46.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.516+0000 7f595e51d640 1 --2- 172.21.15.102:0/1117591245 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5958099110 0x7f595812c820 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f595000d0d0 tx=0x7f595000b990 comp rx=0 tx=0).stop 2024-09-13T17:43:46.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.516+0000 7f595e51d640 1 -- 172.21.15.102:0/1117591245 shutdown_connections 2024-09-13T17:43:46.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.516+0000 7f595e51d640 1 --2- 172.21.15.102:0/1117591245 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5958099110 0x7f595812c820 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:46.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.516+0000 7f595e51d640 1 --2- 172.21.15.102:0/1117591245 >> v2:172.21.15.102:3300/0 conn(0x7f5958098d00 0x7f595812c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:46.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.516+0000 7f595e51d640 1 --2- 172.21.15.102:0/1117591245 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5958096a70 0x7f59580987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:46.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.516+0000 7f595e51d640 1 -- 172.21.15.102:0/1117591245 >> 172.21.15.102:0/1117591245 conn(0x7f595809e720 msgr2=0x7f595809eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:46.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.516+0000 7f595e51d640 1 -- 172.21.15.102:0/1117591245 shutdown_connections 2024-09-13T17:43:46.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.516+0000 7f595e51d640 1 -- 172.21.15.102:0/1117591245 wait complete. 2024-09-13T17:43:46.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.517+0000 7f595e51d640 1 Processor -- start 2024-09-13T17:43:46.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.517+0000 7f595e51d640 1 -- start start 2024-09-13T17:43:46.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.517+0000 7f595e51d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5958096a70 0x7f5958128bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:46.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.517+0000 7f595e51d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5958098d00 0x7f59581290f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:46.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.517+0000 7f595e51d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5958099110 0x7f5958116fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:46.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.517+0000 7f595e51d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595809a410 con 0x7f5958096a70 2024-09-13T17:43:46.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.517+0000 7f595e51d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5958118430 con 0x7f5958098d00 2024-09-13T17:43:46.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.517+0000 7f595e51d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f595811aa30 con 0x7f5958099110 2024-09-13T17:43:46.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.517+0000 7f5957fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5958096a70 0x7f5958128bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:46.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.517+0000 7f59577fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5958098d00 0x7f59581290f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:46.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.517+0000 7f5957fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5958096a70 0x7f5958128bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57684/0 (socket says 172.21.15.102:57684) 2024-09-13T17:43:46.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.517+0000 7f595caba640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5958099110 0x7f5958116fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:46.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.517+0000 7f5957fff640 1 -- 172.21.15.102:0/1054072154 learned_addr learned my addr 172.21.15.102:0/1054072154 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:46.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.518+0000 7f5957fff640 1 -- 172.21.15.102:0/1054072154 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5958099110 msgr2=0x7f5958116fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:46.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.518+0000 7f5957fff640 1 --2- 172.21.15.102:0/1054072154 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5958099110 0x7f5958116fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:46.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.518+0000 7f5957fff640 1 -- 172.21.15.102:0/1054072154 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5958098d00 msgr2=0x7f59581290f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:46.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.518+0000 7f5957fff640 1 --2- 172.21.15.102:0/1054072154 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5958098d00 0x7f59581290f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:46.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.518+0000 7f5957fff640 1 -- 172.21.15.102:0/1054072154 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5948009450 con 0x7f5958096a70 2024-09-13T17:43:46.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.518+0000 7f595caba640 1 --2- 172.21.15.102:0/1054072154 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5958099110 0x7f5958116fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:46.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.518+0000 7f59577fe640 1 --2- 172.21.15.102:0/1054072154 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5958098d00 0x7f59581290f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:46.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.518+0000 7f5957fff640 1 --2- 172.21.15.102:0/1054072154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5958096a70 0x7f5958128bb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f594800ec80 tx=0x7f59480091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:46.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.518+0000 7f593ffff640 1 -- 172.21.15.102:0/1054072154 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594800eec0 con 0x7f5958096a70 2024-09-13T17:43:46.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.518+0000 7f593ffff640 1 -- 172.21.15.102:0/1054072154 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594800cbc0 con 0x7f5958096a70 2024-09-13T17:43:46.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.518+0000 7f595e51d640 1 -- 172.21.15.102:0/1054072154 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f594c009390 con 0x7f5958096a70 2024-09-13T17:43:46.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.518+0000 7f593ffff640 1 -- 172.21.15.102:0/1054072154 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5948013470 con 0x7f5958096a70 2024-09-13T17:43:46.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.518+0000 7f595e51d640 1 -- 172.21.15.102:0/1054072154 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f595000bc30 con 0x7f5958096a70 2024-09-13T17:43:46.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.519+0000 7f593ffff640 1 -- 172.21.15.102:0/1054072154 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f59480137e0 con 0x7f5958096a70 2024-09-13T17:43:46.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.519+0000 7f593ffff640 1 -- 172.21.15.102:0/1054072154 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f5948042530 con 0x7f5958096a70 2024-09-13T17:43:46.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.519+0000 7f595e51d640 1 -- 172.21.15.102:0/1054072154 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f595811a690 con 0x7f5958096a70 2024-09-13T17:43:46.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.525+0000 7f593ffff640 1 -- 172.21.15.102:0/1054072154 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f595811a690 con 0x7f5958096a70 2024-09-13T17:43:46.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.675+0000 7f595e51d640 1 -- 172.21.15.102:0/1054072154 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f595811a690 con 0x7f5958096a70 2024-09-13T17:43:46.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.676+0000 7f593ffff640 1 -- 172.21.15.102:0/1054072154 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f595811a690 con 0x7f5958096a70 2024-09-13T17:43:46.678 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-09-13T17:43:46.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.678+0000 7f595e51d640 1 -- 172.21.15.102:0/1054072154 >> v1:172.21.15.102:6816/34937 conn(0x7f5944031160 legacy=0x7f5944033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:46.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.678+0000 7f595e51d640 1 -- 172.21.15.102:0/1054072154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5958096a70 msgr2=0x7f5958128bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:46.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.678+0000 7f595e51d640 1 --2- 172.21.15.102:0/1054072154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5958096a70 0x7f5958128bb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f594800ec80 tx=0x7f59480091d0 comp rx=0 tx=0).stop 2024-09-13T17:43:46.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.678+0000 7f595e51d640 1 -- 172.21.15.102:0/1054072154 shutdown_connections 2024-09-13T17:43:46.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.678+0000 7f595e51d640 1 --2- 172.21.15.102:0/1054072154 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5958099110 0x7f5958116fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:46.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.678+0000 7f595e51d640 1 --2- 172.21.15.102:0/1054072154 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5958098d00 0x7f59581290f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:46.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.678+0000 7f595e51d640 1 --2- 172.21.15.102:0/1054072154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5958096a70 0x7f5958128bb0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:46.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.678+0000 7f595e51d640 1 -- 172.21.15.102:0/1054072154 >> 172.21.15.102:0/1054072154 conn(0x7f595809e720 msgr2=0x7f59580971f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:46.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.678+0000 7f595e51d640 1 -- 172.21.15.102:0/1054072154 shutdown_connections 2024-09-13T17:43:46.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.678+0000 7f595e51d640 1 -- 172.21.15.102:0/1054072154 wait complete. 2024-09-13T17:43:46.690 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:43:46.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.781+0000 7fd94535d640 1 Processor -- start 2024-09-13T17:43:46.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.781+0000 7fd94535d640 1 -- start start 2024-09-13T17:43:46.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.782+0000 7fd94535d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd940068920 0x7fd9400a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:46.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.782+0000 7fd94535d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9400a1150 0x7fd9400a1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:46.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.782+0000 7fd94535d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd940069260 con 0x7fd94012f440 2024-09-13T17:43:46.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.782+0000 7fd94535d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd940068cf0 con 0x7fd9400a1150 2024-09-13T17:43:46.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.782+0000 7fd94535d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd940069630 con 0x7fd940068920 2024-09-13T17:43:46.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.782+0000 7fd93effd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd940068920 0x7fd9400a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:46.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.782+0000 7fd93e7fc640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9400a1150 0x7fd9400a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:46.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.782+0000 7fd93f7fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fd94012f440 0x7fd94012f8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36400/0 (socket says 172.21.15.102:36400) 2024-09-13T17:43:46.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.782+0000 7fd93f7fe640 1 -- 172.21.15.102:0/1205654220 learned_addr learned my addr 172.21.15.102:0/1205654220 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:46.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.782+0000 7fd93dffb640 1 -- 172.21.15.102:0/1205654220 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 169512913 0 0) 0x7fd940069260 con 0x7fd94012f440 2024-09-13T17:43:46.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.782+0000 7fd93effd640 1 -- 172.21.15.102:0/1205654220 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9400a1150 msgr2=0x7fd9400a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:46.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.782+0000 7fd93effd640 1 --2- 172.21.15.102:0/1205654220 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9400a1150 0x7fd9400a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:46.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.782+0000 7fd93effd640 1 -- 172.21.15.102:0/1205654220 >> v1:172.21.15.102:6789/0 conn(0x7fd94012f440 legacy=0x7fd94012f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:46.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.782+0000 7fd93effd640 1 -- 172.21.15.102:0/1205654220 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd940069870 con 0x7fd940068920 2024-09-13T17:43:46.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.782+0000 7fd93e7fc640 1 --2- 172.21.15.102:0/1205654220 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9400a1150 0x7fd9400a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:46.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.783+0000 7fd93effd640 1 --2- 172.21.15.102:0/1205654220 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd940068920 0x7fd9400a0c10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd930009880 tx=0x7fd9300090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72de95391ab75a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:46.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.783+0000 7fd93dffb640 1 -- 172.21.15.102:0/1205654220 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93002ac50 con 0x7fd940068920 2024-09-13T17:43:46.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.783+0000 7fd93dffb640 1 -- 172.21.15.102:0/1205654220 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93002adb0 con 0x7fd940068920 2024-09-13T17:43:46.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.783+0000 7fd93dffb640 1 -- 172.21.15.102:0/1205654220 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930032670 con 0x7fd940068920 2024-09-13T17:43:46.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.784+0000 7fd94535d640 1 -- 172.21.15.102:0/1205654220 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd940068920 msgr2=0x7fd9400a0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:46.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.784+0000 7fd94535d640 1 --2- 172.21.15.102:0/1205654220 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd940068920 0x7fd9400a0c10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd930009880 tx=0x7fd9300090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:46.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.784+0000 7fd94535d640 1 -- 172.21.15.102:0/1205654220 shutdown_connections 2024-09-13T17:43:46.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.784+0000 7fd94535d640 1 --2- 172.21.15.102:0/1205654220 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd9400a1150 0x7fd9400a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:46.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.784+0000 7fd94535d640 1 --2- 172.21.15.102:0/1205654220 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd940068920 0x7fd9400a0c10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:46.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.784+0000 7fd94535d640 1 -- 172.21.15.102:0/1205654220 >> 172.21.15.102:0/1205654220 conn(0x7fd9400a34c0 msgr2=0x7fd94006e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:46.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.784+0000 7fd94535d640 1 -- 172.21.15.102:0/1205654220 shutdown_connections 2024-09-13T17:43:46.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.784+0000 7fd94535d640 1 -- 172.21.15.102:0/1205654220 wait complete. 2024-09-13T17:43:46.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.785+0000 7fd94535d640 1 Processor -- start 2024-09-13T17:43:46.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.785+0000 7fd94535d640 1 -- start start 2024-09-13T17:43:46.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.785+0000 7fd94535d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd940068920 0x7fd94011ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:46.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.785+0000 7fd94535d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9400a1150 0x7fd94011b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:46.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.785+0000 7fd94535d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd94012f440 0x7fd94011b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:46.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.785+0000 7fd94535d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94009fe90 con 0x7fd9400a1150 2024-09-13T17:43:46.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.785+0000 7fd94535d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9400a1d50 con 0x7fd94012f440 2024-09-13T17:43:46.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.785+0000 7fd94535d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd94011dfd0 con 0x7fd940068920 2024-09-13T17:43:46.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.785+0000 7fd93effd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd940068920 0x7fd94011ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:46.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.785+0000 7fd93e7fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9400a1150 0x7fd94011b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:46.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.785+0000 7fd93f7fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd94012f440 0x7fd94011b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:46.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.785+0000 7fd93e7fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9400a1150 0x7fd94011b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57698/0 (socket says 172.21.15.102:57698) 2024-09-13T17:43:46.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.785+0000 7fd93e7fc640 1 -- 172.21.15.102:0/1151120577 learned_addr learned my addr 172.21.15.102:0/1151120577 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:46.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.785+0000 7fd93effd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd940068920 0x7fd94011ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:43378/0 (socket says 172.21.15.102:43378) 2024-09-13T17:43:46.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.786+0000 7fd93f7fe640 1 -- 172.21.15.102:0/1151120577 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd940068920 msgr2=0x7fd94011ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:46.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.786+0000 7fd93f7fe640 1 --2- 172.21.15.102:0/1151120577 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd940068920 0x7fd94011ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:46.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.786+0000 7fd93f7fe640 1 -- 172.21.15.102:0/1151120577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9400a1150 msgr2=0x7fd94011b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:46.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.786+0000 7fd93f7fe640 1 --2- 172.21.15.102:0/1151120577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9400a1150 0x7fd94011b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:46.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.786+0000 7fd93f7fe640 1 -- 172.21.15.102:0/1151120577 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd934009390 con 0x7fd94012f440 2024-09-13T17:43:46.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.786+0000 7fd93f7fe640 1 --2- 172.21.15.102:0/1151120577 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd94012f440 0x7fd94011b8e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd93800cbe0 tx=0x7fd9380019a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:46.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.786+0000 7fd926ffd640 1 -- 172.21.15.102:0/1151120577 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93800ce00 con 0x7fd94012f440 2024-09-13T17:43:46.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.786+0000 7fd926ffd640 1 -- 172.21.15.102:0/1151120577 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93800a850 con 0x7fd94012f440 2024-09-13T17:43:46.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.786+0000 7fd94535d640 1 -- 172.21.15.102:0/1151120577 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd930009390 con 0x7fd94012f440 2024-09-13T17:43:46.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.786+0000 7fd926ffd640 1 -- 172.21.15.102:0/1151120577 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93800ac90 con 0x7fd94012f440 2024-09-13T17:43:46.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.786+0000 7fd94535d640 1 -- 172.21.15.102:0/1151120577 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd94011cc70 con 0x7fd94012f440 2024-09-13T17:43:46.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.787+0000 7fd926ffd640 1 -- 172.21.15.102:0/1151120577 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fd938007300 con 0x7fd94012f440 2024-09-13T17:43:46.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.787+0000 7fd926ffd640 1 -- 172.21.15.102:0/1151120577 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7fd9380421d0 con 0x7fd94012f440 2024-09-13T17:43:46.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.787+0000 7fd94535d640 1 -- 172.21.15.102:0/1151120577 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd94011dc50 con 0x7fd94012f440 2024-09-13T17:43:46.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.791+0000 7fd926ffd640 1 -- 172.21.15.102:0/1151120577 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fd94011dc50 con 0x7fd94012f440 2024-09-13T17:43:46.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.942+0000 7fd94535d640 1 -- 172.21.15.102:0/1151120577 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd94011dc50 con 0x7fd94012f440 2024-09-13T17:43:46.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.943+0000 7fd926ffd640 1 -- 172.21.15.102:0/1151120577 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+11660 (secure 0 0 0) 0x7fd94011dc50 con 0x7fd94012f440 2024-09-13T17:43:46.945 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:43:46.945 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":23,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:42:48.385896+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:04.672265+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:05.583134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.17","mappings":[{"from":3,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]},{"pgid":"4.25","mappings":[{"from":3,"to":2}]},{"pgid":"4.33","mappings":[{"from":3,"to":0}]},{"pgid":"4.3a","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:43:46.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.945+0000 7fd94535d640 1 -- 172.21.15.102:0/1151120577 >> v1:172.21.15.102:6816/34937 conn(0x7fd92c031160 legacy=0x7fd92c033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:46.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.945+0000 7fd94535d640 1 -- 172.21.15.102:0/1151120577 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd94012f440 msgr2=0x7fd94011b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:46.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.945+0000 7fd94535d640 1 --2- 172.21.15.102:0/1151120577 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd94012f440 0x7fd94011b8e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd93800cbe0 tx=0x7fd9380019a0 comp rx=0 tx=0).stop 2024-09-13T17:43:46.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.945+0000 7fd94535d640 1 -- 172.21.15.102:0/1151120577 shutdown_connections 2024-09-13T17:43:46.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.945+0000 7fd94535d640 1 --2- 172.21.15.102:0/1151120577 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd94012f440 0x7fd94011b8e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:46.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.945+0000 7fd94535d640 1 --2- 172.21.15.102:0/1151120577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9400a1150 0x7fd94011b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:46.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.945+0000 7fd94535d640 1 --2- 172.21.15.102:0/1151120577 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd940068920 0x7fd94011ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:46.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.945+0000 7fd94535d640 1 -- 172.21.15.102:0/1151120577 >> 172.21.15.102:0/1151120577 conn(0x7fd9400a34c0 msgr2=0x7fd940132550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:46.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.945+0000 7fd94535d640 1 -- 172.21.15.102:0/1151120577 shutdown_connections 2024-09-13T17:43:46.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:46.945+0000 7fd94535d640 1 -- 172.21.15.102:0/1151120577 wait complete. 2024-09-13T17:43:46.957 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:54.442612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T17:41:59.076782+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 16, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:42:04.672265+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:42:05.583134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:43:46.958 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:43:47.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.051+0000 7f7123b5d640 1 Processor -- start 2024-09-13T17:43:47.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.051+0000 7f7123b5d640 1 -- start start 2024-09-13T17:43:47.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.052+0000 7f7123b5d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f711c09b340 0x7f711c09b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:47.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.052+0000 7f7123b5d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f711c12c810 0x7f711c137a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:47.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.052+0000 7f7123b5d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f711c069070 con 0x7f711c09bc50 2024-09-13T17:43:47.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.052+0000 7f7123b5d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f711c09cb80 con 0x7f711c09b340 2024-09-13T17:43:47.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.052+0000 7f7123b5d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f711c06a240 con 0x7f711c12c810 2024-09-13T17:43:47.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.052+0000 7f71218f9640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f711c09b340 0x7f711c09b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:47.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.052+0000 7f71210f8640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f711c09bc50 0x7f711c12c100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36408/0 (socket says 172.21.15.102:36408) 2024-09-13T17:43:47.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.052+0000 7f71220fa640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f711c12c810 0x7f711c137a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:47.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.052+0000 7f71218f9640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f711c09b340 0x7f711c09b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57710/0 (socket says 172.21.15.102:57710) 2024-09-13T17:43:47.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.052+0000 7f71210f8640 1 -- 172.21.15.102:0/1895937742 learned_addr learned my addr 172.21.15.102:0/1895937742 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:47.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.052+0000 7f71220fa640 1 -- 172.21.15.102:0/1895937742 >> v2:172.21.15.102:3300/0 conn(0x7f711c09b340 msgr2=0x7f711c09b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:47.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.052+0000 7f71220fa640 1 --2- 172.21.15.102:0/1895937742 >> v2:172.21.15.102:3300/0 conn(0x7f711c09b340 0x7f711c09b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.052+0000 7f71220fa640 1 -- 172.21.15.102:0/1895937742 >> v1:172.21.15.102:6789/0 conn(0x7f711c09bc50 legacy=0x7f711c12c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:47.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.052+0000 7f71220fa640 1 -- 172.21.15.102:0/1895937742 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711c09d240 con 0x7f711c12c810 2024-09-13T17:43:47.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.052+0000 7f71208f7640 1 -- 172.21.15.102:0/1895937742 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1165208422 0 0) 0x7f711c069070 con 0x7f711c09bc50 2024-09-13T17:43:47.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.052+0000 7f71218f9640 1 --2- 172.21.15.102:0/1895937742 >> v2:172.21.15.102:3300/0 conn(0x7f711c09b340 0x7f711c09b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:47.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.053+0000 7f71220fa640 1 --2- 172.21.15.102:0/1895937742 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f711c12c810 0x7f711c137a10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7114009e60 tx=0x7f7114009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be00cce9d519c9ba server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:47.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.053+0000 7f71208f7640 1 -- 172.21.15.102:0/1895937742 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7114007550 con 0x7f711c12c810 2024-09-13T17:43:47.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.053+0000 7f71208f7640 1 -- 172.21.15.102:0/1895937742 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f711402d050 con 0x7f711c12c810 2024-09-13T17:43:47.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.053+0000 7f71208f7640 1 -- 172.21.15.102:0/1895937742 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71140078e0 con 0x7f711c12c810 2024-09-13T17:43:47.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.054+0000 7f7123b5d640 1 -- 172.21.15.102:0/1895937742 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f711c12c810 msgr2=0x7f711c137a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:47.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.054+0000 7f7123b5d640 1 --2- 172.21.15.102:0/1895937742 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f711c12c810 0x7f711c137a10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7114009e60 tx=0x7f7114009780 comp rx=0 tx=0).stop 2024-09-13T17:43:47.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.054+0000 7f7123b5d640 1 -- 172.21.15.102:0/1895937742 shutdown_connections 2024-09-13T17:43:47.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.054+0000 7f7123b5d640 1 --2- 172.21.15.102:0/1895937742 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f711c12c810 0x7f711c137a10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.054+0000 7f7123b5d640 1 --2- 172.21.15.102:0/1895937742 >> v2:172.21.15.102:3300/0 conn(0x7f711c09b340 0x7f711c09b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.054+0000 7f7123b5d640 1 -- 172.21.15.102:0/1895937742 >> 172.21.15.102:0/1895937742 conn(0x7f711c067c00 msgr2=0x7f711c0a0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:47.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.054+0000 7f7123b5d640 1 -- 172.21.15.102:0/1895937742 shutdown_connections 2024-09-13T17:43:47.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.054+0000 7f7123b5d640 1 -- 172.21.15.102:0/1895937742 wait complete. 2024-09-13T17:43:47.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.054+0000 7f7123b5d640 1 Processor -- start 2024-09-13T17:43:47.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.055+0000 7f7123b5d640 1 -- start start 2024-09-13T17:43:47.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.055+0000 7f7123b5d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f711c09b340 0x7f711c11c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:47.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.055+0000 7f7123b5d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f711c09bc50 0x7f711c11c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:47.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.055+0000 7f7123b5d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f711c12c810 0x7f711c11cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:47.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.055+0000 7f7123b5d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711c068680 con 0x7f711c12c810 2024-09-13T17:43:47.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.055+0000 7f7123b5d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f711c12d700 con 0x7f711c09bc50 2024-09-13T17:43:47.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.055+0000 7f7123b5d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f711c11f480 con 0x7f711c09b340 2024-09-13T17:43:47.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.055+0000 7f71218f9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f711c09b340 0x7f711c11c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:47.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.055+0000 7f71210f8640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f711c09bc50 0x7f711c11c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:47.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.055+0000 7f71218f9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f711c09b340 0x7f711c11c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:43392/0 (socket says 172.21.15.102:43392) 2024-09-13T17:43:47.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.055+0000 7f71218f9640 1 -- 172.21.15.102:0/1920759069 learned_addr learned my addr 172.21.15.102:0/1920759069 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:47.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.055+0000 7f71220fa640 1 --2- 172.21.15.102:0/1920759069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f711c12c810 0x7f711c11cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:47.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.055+0000 7f71218f9640 1 -- 172.21.15.102:0/1920759069 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f711c09bc50 msgr2=0x7f711c11c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:47.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.055+0000 7f71218f9640 1 --2- 172.21.15.102:0/1920759069 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f711c09bc50 0x7f711c11c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.055+0000 7f71218f9640 1 -- 172.21.15.102:0/1920759069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f711c12c810 msgr2=0x7f711c11cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:47.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.055+0000 7f71218f9640 1 --2- 172.21.15.102:0/1920759069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f711c12c810 0x7f711c11cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.055+0000 7f71218f9640 1 -- 172.21.15.102:0/1920759069 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7118009390 con 0x7f711c09b340 2024-09-13T17:43:47.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.055+0000 7f71218f9640 1 --2- 172.21.15.102:0/1920759069 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f711c09b340 0x7f711c11c2f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f711800ec80 tx=0x7f71180090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:47.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.056+0000 7f7111ffb640 1 -- 172.21.15.102:0/1920759069 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711800eec0 con 0x7f711c09b340 2024-09-13T17:43:47.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.056+0000 7f7111ffb640 1 -- 172.21.15.102:0/1920759069 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f711800cbc0 con 0x7f711c09b340 2024-09-13T17:43:47.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.056+0000 7f7123b5d640 1 -- 172.21.15.102:0/1920759069 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7114009a20 con 0x7f711c09b340 2024-09-13T17:43:47.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.056+0000 7f7111ffb640 1 -- 172.21.15.102:0/1920759069 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7118013470 con 0x7f711c09b340 2024-09-13T17:43:47.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.056+0000 7f7123b5d640 1 -- 172.21.15.102:0/1920759069 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f711c11fab0 con 0x7f711c09b340 2024-09-13T17:43:47.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.057+0000 7f7111ffb640 1 -- 172.21.15.102:0/1920759069 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f71180137e0 con 0x7f711c09b340 2024-09-13T17:43:47.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.057+0000 7f7111ffb640 1 -- 172.21.15.102:0/1920759069 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f7118042880 con 0x7f711c09b340 2024-09-13T17:43:47.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.057+0000 7f7123b5d640 1 -- 172.21.15.102:0/1920759069 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f711c064000 con 0x7f711c09b340 2024-09-13T17:43:47.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.061+0000 7f7111ffb640 1 -- 172.21.15.102:0/1920759069 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f711801bb90 con 0x7f711c09b340 2024-09-13T17:43:47.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.211+0000 7f7123b5d640 1 -- 172.21.15.102:0/1920759069 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f711c1204f0 con 0x7f711c09b340 2024-09-13T17:43:47.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.211+0000 7f7111ffb640 1 -- 172.21.15.102:0/1920759069 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f711802a0a0 con 0x7f711c09b340 2024-09-13T17:43:47.213 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-09-13T17:43:47.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.213+0000 7f7123b5d640 1 -- 172.21.15.102:0/1920759069 >> v1:172.21.15.102:6816/34937 conn(0x7f7100031160 legacy=0x7f7100033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:47.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.213+0000 7f7123b5d640 1 -- 172.21.15.102:0/1920759069 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f711c09b340 msgr2=0x7f711c11c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:47.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.213+0000 7f7123b5d640 1 --2- 172.21.15.102:0/1920759069 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f711c09b340 0x7f711c11c2f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f711800ec80 tx=0x7f71180090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.213+0000 7f7123b5d640 1 -- 172.21.15.102:0/1920759069 shutdown_connections 2024-09-13T17:43:47.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.213+0000 7f7123b5d640 1 --2- 172.21.15.102:0/1920759069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f711c12c810 0x7f711c11cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.213+0000 7f7123b5d640 1 --2- 172.21.15.102:0/1920759069 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f711c09bc50 0x7f711c11c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.213+0000 7f7123b5d640 1 --2- 172.21.15.102:0/1920759069 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f711c09b340 0x7f711c11c2f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.213+0000 7f7123b5d640 1 -- 172.21.15.102:0/1920759069 >> 172.21.15.102:0/1920759069 conn(0x7f711c067c00 msgr2=0x7f711c06dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:47.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.213+0000 7f7123b5d640 1 -- 172.21.15.102:0/1920759069 shutdown_connections 2024-09-13T17:43:47.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.213+0000 7f7123b5d640 1 -- 172.21.15.102:0/1920759069 wait complete. 2024-09-13T17:43:47.226 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-13T17:43:47.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.320+0000 7f2c0bf9d640 1 Processor -- start 2024-09-13T17:43:47.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.321+0000 7f2c0bf9d640 1 -- start start 2024-09-13T17:43:47.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.321+0000 7f2c0bf9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c04067130 0x7f2c0409db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:47.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.321+0000 7f2c0bf9d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2c0409d6c0 0x7f2c0409e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:47.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.321+0000 7f2c0bf9d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c0409f5d0 con 0x7f2c0412f3f0 2024-09-13T17:43:47.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.321+0000 7f2c0bf9d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c04097800 con 0x7f2c0409d6c0 2024-09-13T17:43:47.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.321+0000 7f2c0bf9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c04096e80 con 0x7f2c04067130 2024-09-13T17:43:47.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.321+0000 7f2c09d39640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c04067130 0x7f2c0409db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:47.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.321+0000 7f2c09538640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2c0409d6c0 0x7f2c0409e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:47.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.321+0000 7f2c0a53a640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f2c0412f3f0 0x7f2c0412f870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36412/0 (socket says 172.21.15.102:36412) 2024-09-13T17:43:47.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.321+0000 7f2c09d39640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c04067130 0x7f2c0409db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:50906/0 (socket says 172.21.15.102:50906) 2024-09-13T17:43:47.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.321+0000 7f2c0a53a640 1 -- 172.21.15.102:0/1144528853 learned_addr learned my addr 172.21.15.102:0/1144528853 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:47.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.322+0000 7f2c09538640 1 -- 172.21.15.102:0/1144528853 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c04067130 msgr2=0x7f2c0409db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:47.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.322+0000 7f2c09538640 1 --2- 172.21.15.102:0/1144528853 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c04067130 0x7f2c0409db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.322+0000 7f2c09538640 1 -- 172.21.15.102:0/1144528853 >> v1:172.21.15.102:6789/0 conn(0x7f2c0412f3f0 legacy=0x7f2c0412f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:47.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.322+0000 7f2c08d37640 1 -- 172.21.15.102:0/1144528853 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3825858164 0 0) 0x7f2c0409f5d0 con 0x7f2c0412f3f0 2024-09-13T17:43:47.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.322+0000 7f2c09538640 1 -- 172.21.15.102:0/1144528853 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c0409e5d0 con 0x7f2c0409d6c0 2024-09-13T17:43:47.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.322+0000 7f2c09d39640 1 --2- 172.21.15.102:0/1144528853 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c04067130 0x7f2c0409db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:47.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.322+0000 7f2c09538640 1 --2- 172.21.15.102:0/1144528853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c0409d6c0 0x7f2c0409e090 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009880 tx=0x7f2bf80090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4176e7f8c5148114 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:47.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.322+0000 7f2c08d37640 1 -- 172.21.15.102:0/1144528853 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf802ace0 con 0x7f2c0409d6c0 2024-09-13T17:43:47.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.322+0000 7f2c08d37640 1 -- 172.21.15.102:0/1144528853 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf802ae40 con 0x7f2c0409d6c0 2024-09-13T17:43:47.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.322+0000 7f2c08d37640 1 -- 172.21.15.102:0/1144528853 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8032670 con 0x7f2c0409d6c0 2024-09-13T17:43:47.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.323+0000 7f2c0bf9d640 1 -- 172.21.15.102:0/1144528853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c0409d6c0 msgr2=0x7f2c0409e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:47.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.323+0000 7f2c0bf9d640 1 --2- 172.21.15.102:0/1144528853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c0409d6c0 0x7f2c0409e090 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009880 tx=0x7f2bf80090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.323+0000 7f2c0bf9d640 1 -- 172.21.15.102:0/1144528853 shutdown_connections 2024-09-13T17:43:47.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.323+0000 7f2c0bf9d640 1 --2- 172.21.15.102:0/1144528853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c0409d6c0 0x7f2c0409e090 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.323+0000 7f2c0bf9d640 1 --2- 172.21.15.102:0/1144528853 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c04067130 0x7f2c0409db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.323+0000 7f2c0bf9d640 1 -- 172.21.15.102:0/1144528853 >> 172.21.15.102:0/1144528853 conn(0x7f2c040a0420 msgr2=0x7f2c0405db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:47.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.323+0000 7f2c0bf9d640 1 -- 172.21.15.102:0/1144528853 shutdown_connections 2024-09-13T17:43:47.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.323+0000 7f2c0bf9d640 1 -- 172.21.15.102:0/1144528853 wait complete. 2024-09-13T17:43:47.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.324+0000 7f2c0bf9d640 1 Processor -- start 2024-09-13T17:43:47.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.324+0000 7f2c0bf9d640 1 -- start start 2024-09-13T17:43:47.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.324+0000 7f2c0bf9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2c04067130 0x7f2c0413eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:47.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.324+0000 7f2c0bf9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c0409d6c0 0x7f2c0413efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:47.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.324+0000 7f2c0bf9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c0412f3f0 0x7f2c0413f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:47.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.324+0000 7f2c0bf9d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0411e9a0 con 0x7f2c0409d6c0 2024-09-13T17:43:47.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.324+0000 7f2c0bf9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0411de60 con 0x7f2c0412f3f0 2024-09-13T17:43:47.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.324+0000 7f2c0bf9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0409a870 con 0x7f2c04067130 2024-09-13T17:43:47.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.324+0000 7f2c09d39640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2c04067130 0x7f2c0413eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:47.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.324+0000 7f2c0a53a640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c0412f3f0 0x7f2c0413f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:47.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.324+0000 7f2c09538640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c0409d6c0 0x7f2c0413efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:47.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.324+0000 7f2c09538640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c0409d6c0 0x7f2c0413efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57726/0 (socket says 172.21.15.102:57726) 2024-09-13T17:43:47.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.324+0000 7f2c09538640 1 -- 172.21.15.102:0/379081558 learned_addr learned my addr 172.21.15.102:0/379081558 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:47.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.325+0000 7f2c09538640 1 -- 172.21.15.102:0/379081558 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2c04067130 msgr2=0x7f2c0413eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:47.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.325+0000 7f2c09538640 1 --2- 172.21.15.102:0/379081558 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2c04067130 0x7f2c0413eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.325+0000 7f2c09538640 1 -- 172.21.15.102:0/379081558 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c0412f3f0 msgr2=0x7f2c0413f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:47.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.325+0000 7f2c09538640 1 --2- 172.21.15.102:0/379081558 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c0412f3f0 0x7f2c0413f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.325+0000 7f2c09538640 1 -- 172.21.15.102:0/379081558 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c00009390 con 0x7f2c0409d6c0 2024-09-13T17:43:47.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.325+0000 7f2c09d39640 1 --2- 172.21.15.102:0/379081558 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2c04067130 0x7f2c0413eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:47.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.325+0000 7f2c0a53a640 1 --2- 172.21.15.102:0/379081558 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c0412f3f0 0x7f2c0413f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:47.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.325+0000 7f2c09538640 1 --2- 172.21.15.102:0/379081558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c0409d6c0 0x7f2c0413efe0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009880 tx=0x7f2bf8032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:47.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.325+0000 7f2bf5ffb640 1 -- 172.21.15.102:0/379081558 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8031780 con 0x7f2c0409d6c0 2024-09-13T17:43:47.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.325+0000 7f2bf5ffb640 1 -- 172.21.15.102:0/379081558 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf80318e0 con 0x7f2c0409d6c0 2024-09-13T17:43:47.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.325+0000 7f2c0bf9d640 1 -- 172.21.15.102:0/379081558 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bf8009390 con 0x7f2c0409d6c0 2024-09-13T17:43:47.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.325+0000 7f2bf5ffb640 1 -- 172.21.15.102:0/379081558 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8031ca0 con 0x7f2c0409d6c0 2024-09-13T17:43:47.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.325+0000 7f2c0bf9d640 1 -- 172.21.15.102:0/379081558 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c0411f8c0 con 0x7f2c0409d6c0 2024-09-13T17:43:47.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.326+0000 7f2bf5ffb640 1 -- 172.21.15.102:0/379081558 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f2bf8039040 con 0x7f2c0409d6c0 2024-09-13T17:43:47.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.326+0000 7f2bf5ffb640 1 -- 172.21.15.102:0/379081558 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f2bf8062410 con 0x7f2c0409d6c0 2024-09-13T17:43:47.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.326+0000 7f2c0bf9d640 1 -- 172.21.15.102:0/379081558 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c0411eb30 con 0x7f2c0409d6c0 2024-09-13T17:43:47.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.331+0000 7f2bf5ffb640 1 -- 172.21.15.102:0/379081558 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f2bf8049080 con 0x7f2c0409d6c0 2024-09-13T17:43:47.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.483+0000 7f2c0bf9d640 1 -- 172.21.15.102:0/379081558 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2c04064000 con 0x7f2c0409d6c0 2024-09-13T17:43:47.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.483+0000 7f2bf5ffb640 1 -- 172.21.15.102:0/379081558 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2bf8037020 con 0x7f2c0409d6c0 2024-09-13T17:43:47.485 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-09-13T17:43:47.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.485+0000 7f2c0bf9d640 1 -- 172.21.15.102:0/379081558 >> v1:172.21.15.102:6816/34937 conn(0x7f2be8031160 legacy=0x7f2be8033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:47.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.485+0000 7f2c0bf9d640 1 -- 172.21.15.102:0/379081558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c0409d6c0 msgr2=0x7f2c0413efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:47.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.485+0000 7f2c0bf9d640 1 --2- 172.21.15.102:0/379081558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c0409d6c0 0x7f2c0413efe0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009880 tx=0x7f2bf8032000 comp rx=0 tx=0).stop 2024-09-13T17:43:47.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.485+0000 7f2c0bf9d640 1 -- 172.21.15.102:0/379081558 shutdown_connections 2024-09-13T17:43:47.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.485+0000 7f2c0bf9d640 1 --2- 172.21.15.102:0/379081558 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2c0412f3f0 0x7f2c0413f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.485+0000 7f2c0bf9d640 1 --2- 172.21.15.102:0/379081558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c0409d6c0 0x7f2c0413efe0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.485+0000 7f2c0bf9d640 1 --2- 172.21.15.102:0/379081558 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2c04067130 0x7f2c0413eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.485+0000 7f2c0bf9d640 1 -- 172.21.15.102:0/379081558 >> 172.21.15.102:0/379081558 conn(0x7f2c040a0420 msgr2=0x7f2c040988f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:47.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.485+0000 7f2c0bf9d640 1 -- 172.21.15.102:0/379081558 shutdown_connections 2024-09-13T17:43:47.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.485+0000 7f2c0bf9d640 1 -- 172.21.15.102:0/379081558 wait complete. 2024-09-13T17:43:47.497 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:43:47.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.593+0000 7ffa5299d640 1 Processor -- start 2024-09-13T17:43:47.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.593+0000 7ffa5299d640 1 -- start start 2024-09-13T17:43:47.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.593+0000 7ffa5299d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa4c096a70 0x7ffa4c12c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:47.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.593+0000 7ffa5299d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ffa4c12c900 0x7ffa4c098410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:47.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.593+0000 7ffa5299d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffa4c12ccd0 0x7ffa4c098950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:47.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.593+0000 7ffa5299d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa4c096e40 con 0x7ffa4c12c900 2024-09-13T17:43:47.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.593+0000 7ffa5299d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c099540 con 0x7ffa4c12ccd0 2024-09-13T17:43:47.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.593+0000 7ffa5299d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c09a8e0 con 0x7ffa4c096a70 2024-09-13T17:43:47.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.593+0000 7ffa50f3a640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffa4c12ccd0 0x7ffa4c098950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:47.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.593+0000 7ffa4b7fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ffa4c12c900 0x7ffa4c098410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:47.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.593+0000 7ffa50f3a640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffa4c12ccd0 0x7ffa4c098950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:50926/0 (socket says 172.21.15.102:50926) 2024-09-13T17:43:47.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.593+0000 7ffa4bfff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa4c096a70 0x7ffa4c12c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:47.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.593+0000 7ffa50f3a640 1 -- 172.21.15.102:0/834946366 learned_addr learned my addr 172.21.15.102:0/834946366 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:47.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.594+0000 7ffa4bfff640 1 -- 172.21.15.102:0/834946366 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffa4c12ccd0 msgr2=0x7ffa4c098950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:47.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.594+0000 7ffa4bfff640 1 --2- 172.21.15.102:0/834946366 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffa4c12ccd0 0x7ffa4c098950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.594+0000 7ffa4bfff640 1 -- 172.21.15.102:0/834946366 >> v2:172.21.15.102:3300/0 conn(0x7ffa4c12c900 msgr2=0x7ffa4c098410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:47.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.594+0000 7ffa4bfff640 1 --2- 172.21.15.102:0/834946366 >> v2:172.21.15.102:3300/0 conn(0x7ffa4c12c900 0x7ffa4c098410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.594+0000 7ffa4bfff640 1 -- 172.21.15.102:0/834946366 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa4c099760 con 0x7ffa4c096a70 2024-09-13T17:43:47.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.594+0000 7ffa4b7fe640 1 --2- 172.21.15.102:0/834946366 >> v2:172.21.15.102:3300/0 conn(0x7ffa4c12c900 0x7ffa4c098410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:47.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.594+0000 7ffa50f3a640 1 --2- 172.21.15.102:0/834946366 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffa4c12ccd0 0x7ffa4c098950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:47.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.594+0000 7ffa4bfff640 1 --2- 172.21.15.102:0/834946366 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa4c096a70 0x7ffa4c12c3c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c009880 tx=0x7ffa3c0090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52e5f733a3fa2b04 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:47.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.595+0000 7ffa4affd640 1 -- 172.21.15.102:0/834946366 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c02ac40 con 0x7ffa4c096a70 2024-09-13T17:43:47.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.595+0000 7ffa4affd640 1 -- 172.21.15.102:0/834946366 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3c02ada0 con 0x7ffa4c096a70 2024-09-13T17:43:47.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.595+0000 7ffa4affd640 1 -- 172.21.15.102:0/834946366 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c032670 con 0x7ffa4c096a70 2024-09-13T17:43:47.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.595+0000 7ffa5299d640 1 -- 172.21.15.102:0/834946366 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa4c096a70 msgr2=0x7ffa4c12c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:47.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.595+0000 7ffa5299d640 1 --2- 172.21.15.102:0/834946366 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa4c096a70 0x7ffa4c12c3c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c009880 tx=0x7ffa3c0090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.595+0000 7ffa5299d640 1 -- 172.21.15.102:0/834946366 shutdown_connections 2024-09-13T17:43:47.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.595+0000 7ffa5299d640 1 --2- 172.21.15.102:0/834946366 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffa4c12ccd0 0x7ffa4c098950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.595+0000 7ffa5299d640 1 --2- 172.21.15.102:0/834946366 >> v2:172.21.15.102:3300/0 conn(0x7ffa4c12c900 0x7ffa4c098410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.595+0000 7ffa5299d640 1 --2- 172.21.15.102:0/834946366 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa4c096a70 0x7ffa4c12c3c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.595+0000 7ffa5299d640 1 -- 172.21.15.102:0/834946366 >> 172.21.15.102:0/834946366 conn(0x7ffa4c09e6b0 msgr2=0x7ffa4c09eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:47.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.595+0000 7ffa5299d640 1 -- 172.21.15.102:0/834946366 shutdown_connections 2024-09-13T17:43:47.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.595+0000 7ffa5299d640 1 -- 172.21.15.102:0/834946366 wait complete. 2024-09-13T17:43:47.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.596+0000 7ffa5299d640 1 Processor -- start 2024-09-13T17:43:47.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.596+0000 7ffa5299d640 1 -- start start 2024-09-13T17:43:47.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.597+0000 7ffa5299d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa4c096a70 0x7ffa4c13c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:47.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.597+0000 7ffa5299d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa4c12c900 0x7ffa4c13cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:47.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.597+0000 7ffa5299d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffa4c12ccd0 0x7ffa4c13d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:47.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.597+0000 7ffa5299d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c097760 con 0x7ffa4c12c900 2024-09-13T17:43:47.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.597+0000 7ffa5299d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c09dc80 con 0x7ffa4c12ccd0 2024-09-13T17:43:47.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.597+0000 7ffa5299d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c09a030 con 0x7ffa4c096a70 2024-09-13T17:43:47.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.597+0000 7ffa4bfff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa4c096a70 0x7ffa4c13c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:47.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.597+0000 7ffa50f3a640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffa4c12ccd0 0x7ffa4c13d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:47.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.597+0000 7ffa4bfff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa4c096a70 0x7ffa4c13c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:43412/0 (socket says 172.21.15.102:43412) 2024-09-13T17:43:47.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.597+0000 7ffa4bfff640 1 -- 172.21.15.102:0/207348650 learned_addr learned my addr 172.21.15.102:0/207348650 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:47.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.597+0000 7ffa4b7fe640 1 --2- 172.21.15.102:0/207348650 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa4c12c900 0x7ffa4c13cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:47.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.597+0000 7ffa50f3a640 1 -- 172.21.15.102:0/207348650 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa4c096a70 msgr2=0x7ffa4c13c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:47.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.597+0000 7ffa50f3a640 1 --2- 172.21.15.102:0/207348650 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa4c096a70 0x7ffa4c13c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.597+0000 7ffa50f3a640 1 -- 172.21.15.102:0/207348650 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa4c12c900 msgr2=0x7ffa4c13cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:47.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.597+0000 7ffa50f3a640 1 --2- 172.21.15.102:0/207348650 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa4c12c900 0x7ffa4c13cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.597+0000 7ffa50f3a640 1 -- 172.21.15.102:0/207348650 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa40009390 con 0x7ffa4c12ccd0 2024-09-13T17:43:47.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.597+0000 7ffa50f3a640 1 --2- 172.21.15.102:0/207348650 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffa4c12ccd0 0x7ffa4c13d440 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009ee0 tx=0x7ffa4400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:47.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.597+0000 7ffa33fff640 1 -- 172.21.15.102:0/207348650 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa44018070 con 0x7ffa4c12ccd0 2024-09-13T17:43:47.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.597+0000 7ffa33fff640 1 -- 172.21.15.102:0/207348650 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa4400f040 con 0x7ffa4c12ccd0 2024-09-13T17:43:47.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.597+0000 7ffa5299d640 1 -- 172.21.15.102:0/207348650 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa44009ae0 con 0x7ffa4c12ccd0 2024-09-13T17:43:47.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.597+0000 7ffa33fff640 1 -- 172.21.15.102:0/207348650 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa440092e0 con 0x7ffa4c12ccd0 2024-09-13T17:43:47.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.597+0000 7ffa5299d640 1 -- 172.21.15.102:0/207348650 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa3c009390 con 0x7ffa4c12ccd0 2024-09-13T17:43:47.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.598+0000 7ffa33fff640 1 -- 172.21.15.102:0/207348650 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7ffa44003020 con 0x7ffa4c12ccd0 2024-09-13T17:43:47.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.598+0000 7ffa33fff640 1 -- 172.21.15.102:0/207348650 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7ffa44042070 con 0x7ffa4c12ccd0 2024-09-13T17:43:47.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.598+0000 7ffa5299d640 1 -- 172.21.15.102:0/207348650 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa4c064000 con 0x7ffa4c12ccd0 2024-09-13T17:43:47.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.603+0000 7ffa33fff640 1 -- 172.21.15.102:0/207348650 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7ffa4401d410 con 0x7ffa4c12ccd0 2024-09-13T17:43:47.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.755+0000 7ffa5299d640 1 -- 172.21.15.102:0/207348650 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffa4c118810 con 0x7ffa4c12ccd0 2024-09-13T17:43:47.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.756+0000 7ffa33fff640 1 -- 172.21.15.102:0/207348650 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffa440280a0 con 0x7ffa4c12ccd0 2024-09-13T17:43:47.758 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-09-13T17:43:47.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.758+0000 7ffa5299d640 1 -- 172.21.15.102:0/207348650 >> v1:172.21.15.102:6816/34937 conn(0x7ffa38031160 legacy=0x7ffa38033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:47.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.758+0000 7ffa5299d640 1 -- 172.21.15.102:0/207348650 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffa4c12ccd0 msgr2=0x7ffa4c13d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:47.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.758+0000 7ffa5299d640 1 --2- 172.21.15.102:0/207348650 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffa4c12ccd0 0x7ffa4c13d440 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009ee0 tx=0x7ffa4400b040 comp rx=0 tx=0).stop 2024-09-13T17:43:47.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.758+0000 7ffa5299d640 1 -- 172.21.15.102:0/207348650 shutdown_connections 2024-09-13T17:43:47.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.758+0000 7ffa5299d640 1 --2- 172.21.15.102:0/207348650 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ffa4c12ccd0 0x7ffa4c13d440 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.758+0000 7ffa5299d640 1 --2- 172.21.15.102:0/207348650 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa4c12c900 0x7ffa4c13cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.758+0000 7ffa5299d640 1 --2- 172.21.15.102:0/207348650 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ffa4c096a70 0x7ffa4c13c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.758+0000 7ffa5299d640 1 -- 172.21.15.102:0/207348650 >> 172.21.15.102:0/207348650 conn(0x7ffa4c09e6b0 msgr2=0x7ffa4c12b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:47.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.758+0000 7ffa5299d640 1 -- 172.21.15.102:0/207348650 shutdown_connections 2024-09-13T17:43:47.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.758+0000 7ffa5299d640 1 -- 172.21.15.102:0/207348650 wait complete. 2024-09-13T17:43:47.770 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:43:47.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.864+0000 7f325ab1d640 1 Processor -- start 2024-09-13T17:43:47.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.864+0000 7f325ab1d640 1 -- start start 2024-09-13T17:43:47.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.864+0000 7f325ab1d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3254096df0 0x7f32540971c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:47.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.864+0000 7f325ab1d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3254097d30 0x7f325412d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:47.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.864+0000 7f325ab1d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3254097700 0x7f325412d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:47.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.864+0000 7f325ab1d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3254098290 con 0x7f3254096df0 2024-09-13T17:43:47.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.864+0000 7f325ab1d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f325406a850 con 0x7f3254097700 2024-09-13T17:43:47.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.864+0000 7f325ab1d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3254096bf0 con 0x7f3254097d30 2024-09-13T17:43:47.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.864+0000 7f32588b9640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3254096df0 0x7f32540971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:47.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.864+0000 7f32590ba640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3254097700 0x7f325412d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:47.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.864+0000 7f32590ba640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3254097700 0x7f325412d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:50950/0 (socket says 172.21.15.102:50950) 2024-09-13T17:43:47.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.864+0000 7f32588b9640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3254096df0 0x7f32540971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57762/0 (socket says 172.21.15.102:57762) 2024-09-13T17:43:47.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.864+0000 7f32590ba640 1 -- 172.21.15.102:0/308443733 learned_addr learned my addr 172.21.15.102:0/308443733 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:47.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.864+0000 7f3253fff640 1 --2- 172.21.15.102:0/308443733 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3254097d30 0x7f325412d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:47.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.865+0000 7f32590ba640 1 -- 172.21.15.102:0/308443733 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3254097d30 msgr2=0x7f325412d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:47.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.865+0000 7f32590ba640 1 --2- 172.21.15.102:0/308443733 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3254097d30 0x7f325412d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.865+0000 7f32590ba640 1 -- 172.21.15.102:0/308443733 >> v2:172.21.15.102:3300/0 conn(0x7f3254096df0 msgr2=0x7f32540971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:47.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.865+0000 7f32590ba640 1 --2- 172.21.15.102:0/308443733 >> v2:172.21.15.102:3300/0 conn(0x7f3254096df0 0x7f32540971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.865+0000 7f32590ba640 1 -- 172.21.15.102:0/308443733 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f325406a590 con 0x7f3254097700 2024-09-13T17:43:47.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.865+0000 7f32588b9640 1 --2- 172.21.15.102:0/308443733 >> v2:172.21.15.102:3300/0 conn(0x7f3254096df0 0x7f32540971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:47.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.865+0000 7f3253fff640 1 --2- 172.21.15.102:0/308443733 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3254097d30 0x7f325412d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:47.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.865+0000 7f32590ba640 1 --2- 172.21.15.102:0/308443733 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3254097700 0x7f325412d7c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f323c009940 tx=0x7f323c0091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1e36cd28c709421 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:47.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.866+0000 7f32537fe640 1 -- 172.21.15.102:0/308443733 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c02ad20 con 0x7f3254097700 2024-09-13T17:43:47.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.866+0000 7f32537fe640 1 -- 172.21.15.102:0/308443733 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323c02ae80 con 0x7f3254097700 2024-09-13T17:43:47.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.866+0000 7f32537fe640 1 -- 172.21.15.102:0/308443733 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c032670 con 0x7f3254097700 2024-09-13T17:43:47.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.866+0000 7f325ab1d640 1 -- 172.21.15.102:0/308443733 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3254097700 msgr2=0x7f325412d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:47.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.866+0000 7f325ab1d640 1 --2- 172.21.15.102:0/308443733 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3254097700 0x7f325412d7c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f323c009940 tx=0x7f323c0091d0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.866+0000 7f325ab1d640 1 -- 172.21.15.102:0/308443733 shutdown_connections 2024-09-13T17:43:47.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.866+0000 7f325ab1d640 1 --2- 172.21.15.102:0/308443733 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3254097700 0x7f325412d7c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.866+0000 7f325ab1d640 1 --2- 172.21.15.102:0/308443733 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3254097d30 0x7f325412d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.866+0000 7f325ab1d640 1 --2- 172.21.15.102:0/308443733 >> v2:172.21.15.102:3300/0 conn(0x7f3254096df0 0x7f32540971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.866+0000 7f325ab1d640 1 -- 172.21.15.102:0/308443733 >> 172.21.15.102:0/308443733 conn(0x7f325409e720 msgr2=0x7f325409eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:47.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.866+0000 7f325ab1d640 1 -- 172.21.15.102:0/308443733 shutdown_connections 2024-09-13T17:43:47.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.866+0000 7f325ab1d640 1 -- 172.21.15.102:0/308443733 wait complete. 2024-09-13T17:43:47.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.867+0000 7f325ab1d640 1 Processor -- start 2024-09-13T17:43:47.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.867+0000 7f325ab1d640 1 -- start start 2024-09-13T17:43:47.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.867+0000 7f325ab1d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3254096df0 0x7f3254117ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:47.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.867+0000 7f325ab1d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3254097700 0x7f32541181e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:47.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.867+0000 7f325ab1d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3254097d30 0x7f325413a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:47.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.867+0000 7f325ab1d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3254119a20 con 0x7f3254096df0 2024-09-13T17:43:47.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.867+0000 7f325ab1d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3254069df0 con 0x7f3254097d30 2024-09-13T17:43:47.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.867+0000 7f325ab1d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f325411a8f0 con 0x7f3254097700 2024-09-13T17:43:47.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.868+0000 7f32588b9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3254096df0 0x7f3254117ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:47.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.868+0000 7f3253fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3254097700 0x7f32541181e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:47.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.868+0000 7f32590ba640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3254097d30 0x7f325413a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:47.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.868+0000 7f32588b9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3254096df0 0x7f3254117ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57770/0 (socket says 172.21.15.102:57770) 2024-09-13T17:43:47.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.868+0000 7f32588b9640 1 -- 172.21.15.102:0/3691249238 learned_addr learned my addr 172.21.15.102:0/3691249238 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:47.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.868+0000 7f3253fff640 1 -- 172.21.15.102:0/3691249238 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3254097d30 msgr2=0x7f325413a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:47.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.868+0000 7f3253fff640 1 --2- 172.21.15.102:0/3691249238 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3254097d30 0x7f325413a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.868+0000 7f3253fff640 1 -- 172.21.15.102:0/3691249238 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3254096df0 msgr2=0x7f3254117ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:47.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.868+0000 7f3253fff640 1 --2- 172.21.15.102:0/3691249238 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3254096df0 0x7f3254117ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.868+0000 7f3253fff640 1 -- 172.21.15.102:0/3691249238 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f324c009390 con 0x7f3254097700 2024-09-13T17:43:47.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.868+0000 7f32590ba640 1 --2- 172.21.15.102:0/3691249238 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3254097d30 0x7f325413a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:47.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.868+0000 7f32588b9640 1 --2- 172.21.15.102:0/3691249238 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3254096df0 0x7f3254117ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:47.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.868+0000 7f3253fff640 1 --2- 172.21.15.102:0/3691249238 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3254097700 0x7f32541181e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3244011310 tx=0x7f324400b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:47.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.868+0000 7f3250ff9640 1 -- 172.21.15.102:0/3691249238 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3244011ee0 con 0x7f3254097700 2024-09-13T17:43:47.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.868+0000 7f3250ff9640 1 -- 172.21.15.102:0/3691249238 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3244014610 con 0x7f3254097700 2024-09-13T17:43:47.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.868+0000 7f325ab1d640 1 -- 172.21.15.102:0/3691249238 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f324400bb50 con 0x7f3254097700 2024-09-13T17:43:47.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.868+0000 7f3250ff9640 1 -- 172.21.15.102:0/3691249238 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32440149b0 con 0x7f3254097700 2024-09-13T17:43:47.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.868+0000 7f325ab1d640 1 -- 172.21.15.102:0/3691249238 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323c009450 con 0x7f3254097700 2024-09-13T17:43:47.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.869+0000 7f3250ff9640 1 -- 172.21.15.102:0/3691249238 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f324401c3f0 con 0x7f3254097700 2024-09-13T17:43:47.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.869+0000 7f3250ff9640 1 -- 172.21.15.102:0/3691249238 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f324400c040 con 0x7f3254097700 2024-09-13T17:43:47.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.869+0000 7f325ab1d640 1 -- 172.21.15.102:0/3691249238 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3254064000 con 0x7f3254097700 2024-09-13T17:43:47.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:47.874+0000 7f3250ff9640 1 -- 172.21.15.102:0/3691249238 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f3244014da0 con 0x7f3254097700 2024-09-13T17:43:48.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.025+0000 7f325ab1d640 1 -- 172.21.15.102:0/3691249238 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f32540994f0 con 0x7f3254097700 2024-09-13T17:43:48.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.025+0000 7f3250ff9640 1 -- 172.21.15.102:0/3691249238 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f324401a300 con 0x7f3254097700 2024-09-13T17:43:48.027 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-09-13T17:43:48.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.027+0000 7f325ab1d640 1 -- 172.21.15.102:0/3691249238 >> v1:172.21.15.102:6816/34937 conn(0x7f3234031160 legacy=0x7f3234033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:48.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.027+0000 7f325ab1d640 1 -- 172.21.15.102:0/3691249238 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3254097700 msgr2=0x7f32541181e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:48.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.027+0000 7f325ab1d640 1 --2- 172.21.15.102:0/3691249238 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3254097700 0x7f32541181e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3244011310 tx=0x7f324400b8b0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.027+0000 7f325ab1d640 1 -- 172.21.15.102:0/3691249238 shutdown_connections 2024-09-13T17:43:48.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.027+0000 7f325ab1d640 1 --2- 172.21.15.102:0/3691249238 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3254097d30 0x7f325413a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.027+0000 7f325ab1d640 1 --2- 172.21.15.102:0/3691249238 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3254097700 0x7f32541181e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.027+0000 7f325ab1d640 1 --2- 172.21.15.102:0/3691249238 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3254096df0 0x7f3254117ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.027+0000 7f325ab1d640 1 -- 172.21.15.102:0/3691249238 >> 172.21.15.102:0/3691249238 conn(0x7f325409e720 msgr2=0x7f325409d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:48.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.027+0000 7f325ab1d640 1 -- 172.21.15.102:0/3691249238 shutdown_connections 2024-09-13T17:43:48.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.027+0000 7f325ab1d640 1 -- 172.21.15.102:0/3691249238 wait complete. 2024-09-13T17:43:48.048 INFO:tasks.cephfs_test_runner:Starting test: test_newops_getvxattr (tasks.cephfs.test_newops.TestNewOps) For nautilus it will crash the MDSs when receive unknown OPs, as a workaround 2024-09-13T17:43:48.048 INFO:tasks.cephfs_test_runner:test_newops_getvxattr (tasks.cephfs.test_newops.TestNewOps) 2024-09-13T17:43:48.049 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:43:48.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.141+0000 7fa151d9d640 1 Processor -- start 2024-09-13T17:43:48.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.141+0000 7fa151d9d640 1 -- start start 2024-09-13T17:43:48.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.142+0000 7fa151d9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa14c128330 0x7fa14c128700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:48.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.142+0000 7fa151d9d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa14c128c40 0x7fa14c12b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:48.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.142+0000 7fa151d9d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa14c097be0 con 0x7fa14c12bd80 2024-09-13T17:43:48.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.142+0000 7fa151d9d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa14c098e70 con 0x7fa14c128c40 2024-09-13T17:43:48.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.142+0000 7fa151d9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c098310 con 0x7fa14c128330 2024-09-13T17:43:48.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.142+0000 7fa14b7fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa14c128330 0x7fa14c128700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:48.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.142+0000 7fa14b7fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa14c128330 0x7fa14c128700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:50956/0 (socket says 172.21.15.102:50956) 2024-09-13T17:43:48.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.142+0000 7fa14b7fe640 1 -- 172.21.15.102:0/1122050722 learned_addr learned my addr 172.21.15.102:0/1122050722 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:48.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.142+0000 7fa14affd640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa14c128c40 0x7fa14c12b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:48.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.142+0000 7fa14b7fe640 1 -- 172.21.15.102:0/1122050722 >> v2:172.21.15.102:3300/0 conn(0x7fa14c128c40 msgr2=0x7fa14c12b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:48.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.142+0000 7fa14b7fe640 1 --2- 172.21.15.102:0/1122050722 >> v2:172.21.15.102:3300/0 conn(0x7fa14c128c40 0x7fa14c12b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.142+0000 7fa14b7fe640 1 -- 172.21.15.102:0/1122050722 >> v1:172.21.15.102:6789/0 conn(0x7fa14c12bd80 legacy=0x7fa14c132b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:48.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.142+0000 7fa14b7fe640 1 -- 172.21.15.102:0/1122050722 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14c0985d0 con 0x7fa14c128330 2024-09-13T17:43:48.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.142+0000 7fa14a7fc640 1 -- 172.21.15.102:0/1122050722 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3834006872 0 0) 0x7fa14c097be0 con 0x7fa14c12bd80 2024-09-13T17:43:48.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.142+0000 7fa14affd640 1 --2- 172.21.15.102:0/1122050722 >> v2:172.21.15.102:3300/0 conn(0x7fa14c128c40 0x7fa14c12b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:48.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.143+0000 7fa14b7fe640 1 --2- 172.21.15.102:0/1122050722 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa14c128330 0x7fa14c128700 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa13c009910 tx=0x7fa13c0091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c71faa42c34a9b76 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:48.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.143+0000 7fa14a7fc640 1 -- 172.21.15.102:0/1122050722 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c02ae40 con 0x7fa14c128330 2024-09-13T17:43:48.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.143+0000 7fa14a7fc640 1 -- 172.21.15.102:0/1122050722 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13c032440 con 0x7fa14c128330 2024-09-13T17:43:48.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.143+0000 7fa14a7fc640 1 -- 172.21.15.102:0/1122050722 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c0327e0 con 0x7fa14c128330 2024-09-13T17:43:48.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.144+0000 7fa151d9d640 1 -- 172.21.15.102:0/1122050722 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa14c128330 msgr2=0x7fa14c128700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:48.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.144+0000 7fa151d9d640 1 --2- 172.21.15.102:0/1122050722 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa14c128330 0x7fa14c128700 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa13c009910 tx=0x7fa13c0091d0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.144+0000 7fa151d9d640 1 -- 172.21.15.102:0/1122050722 shutdown_connections 2024-09-13T17:43:48.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.144+0000 7fa151d9d640 1 --2- 172.21.15.102:0/1122050722 >> v2:172.21.15.102:3300/0 conn(0x7fa14c128c40 0x7fa14c12b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.144+0000 7fa151d9d640 1 --2- 172.21.15.102:0/1122050722 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa14c128330 0x7fa14c128700 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.144+0000 7fa151d9d640 1 -- 172.21.15.102:0/1122050722 >> 172.21.15.102:0/1122050722 conn(0x7fa14c09e820 msgr2=0x7fa14c09ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:48.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.144+0000 7fa151d9d640 1 -- 172.21.15.102:0/1122050722 shutdown_connections 2024-09-13T17:43:48.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.144+0000 7fa151d9d640 1 -- 172.21.15.102:0/1122050722 wait complete. 2024-09-13T17:43:48.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.145+0000 7fa151d9d640 1 Processor -- start 2024-09-13T17:43:48.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.145+0000 7fa151d9d640 1 -- start start 2024-09-13T17:43:48.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.145+0000 7fa151d9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa14c128330 0x7fa14c114e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:48.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.145+0000 7fa151d9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa14c128c40 0x7fa14c1153c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:48.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.145+0000 7fa151d9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa14c12bd80 0x7fa14c115900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:48.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.145+0000 7fa151d9d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c117280 con 0x7fa14c12bd80 2024-09-13T17:43:48.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.145+0000 7fa151d9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c116f30 con 0x7fa14c128c40 2024-09-13T17:43:48.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.145+0000 7fa151d9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c125f20 con 0x7fa14c128330 2024-09-13T17:43:48.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.145+0000 7fa14b7fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa14c128330 0x7fa14c114e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:48.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.145+0000 7fa14affd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa14c128c40 0x7fa14c1153c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:48.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.145+0000 7fa14bfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa14c12bd80 0x7fa14c115900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:48.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.145+0000 7fa14b7fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa14c128330 0x7fa14c114e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:43428/0 (socket says 172.21.15.102:43428) 2024-09-13T17:43:48.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.145+0000 7fa14b7fe640 1 -- 172.21.15.102:0/1631929607 learned_addr learned my addr 172.21.15.102:0/1631929607 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:48.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.145+0000 7fa14affd640 1 -- 172.21.15.102:0/1631929607 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa14c128330 msgr2=0x7fa14c114e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:48.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.145+0000 7fa14affd640 1 --2- 172.21.15.102:0/1631929607 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa14c128330 0x7fa14c114e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.145+0000 7fa14affd640 1 -- 172.21.15.102:0/1631929607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa14c12bd80 msgr2=0x7fa14c115900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:48.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.145+0000 7fa14affd640 1 --2- 172.21.15.102:0/1631929607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa14c12bd80 0x7fa14c115900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.145+0000 7fa14affd640 1 -- 172.21.15.102:0/1631929607 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa140009390 con 0x7fa14c128c40 2024-09-13T17:43:48.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.145+0000 7fa14b7fe640 1 --2- 172.21.15.102:0/1631929607 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa14c128330 0x7fa14c114e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:48.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.145+0000 7fa14bfff640 1 --2- 172.21.15.102:0/1631929607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa14c12bd80 0x7fa14c115900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:48.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.146+0000 7fa14affd640 1 --2- 172.21.15.102:0/1631929607 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa14c128c40 0x7fa14c1153c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa14000ea20 tx=0x7fa1400090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:48.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.146+0000 7fa1337fe640 1 -- 172.21.15.102:0/1631929607 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14000ca10 con 0x7fa14c128c40 2024-09-13T17:43:48.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.146+0000 7fa1337fe640 1 -- 172.21.15.102:0/1631929607 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14000cb70 con 0x7fa14c128c40 2024-09-13T17:43:48.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.146+0000 7fa151d9d640 1 -- 172.21.15.102:0/1631929607 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa13c009450 con 0x7fa14c128c40 2024-09-13T17:43:48.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.146+0000 7fa1337fe640 1 -- 172.21.15.102:0/1631929607 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa140013470 con 0x7fa14c128c40 2024-09-13T17:43:48.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.146+0000 7fa151d9d640 1 -- 172.21.15.102:0/1631929607 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa14c06ea40 con 0x7fa14c128c40 2024-09-13T17:43:48.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.147+0000 7fa1337fe640 1 -- 172.21.15.102:0/1631929607 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fa1400137e0 con 0x7fa14c128c40 2024-09-13T17:43:48.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.147+0000 7fa151d9d640 1 -- 172.21.15.102:0/1631929607 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa14c064000 con 0x7fa14c128c40 2024-09-13T17:43:48.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.147+0000 7fa1337fe640 1 -- 172.21.15.102:0/1631929607 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7fa140042590 con 0x7fa14c128c40 2024-09-13T17:43:48.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.151+0000 7fa1337fe640 1 -- 172.21.15.102:0/1631929607 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fa1400236b0 con 0x7fa14c128c40 2024-09-13T17:43:48.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.305+0000 7fa151d9d640 1 -- 172.21.15.102:0/1631929607 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa14c067690 con 0x7fa14c128c40 2024-09-13T17:43:48.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.306+0000 7fa1337fe640 1 -- 172.21.15.102:0/1631929607 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+11660 (secure 0 0 0) 0x7fa1400232f0 con 0x7fa14c128c40 2024-09-13T17:43:48.308 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:43:48.308 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":23,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:42:48.385896+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:04.672265+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:05.583134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.17","mappings":[{"from":3,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]},{"pgid":"4.25","mappings":[{"from":3,"to":2}]},{"pgid":"4.33","mappings":[{"from":3,"to":0}]},{"pgid":"4.3a","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:43:48.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.308+0000 7fa151d9d640 1 -- 172.21.15.102:0/1631929607 >> v1:172.21.15.102:6816/34937 conn(0x7fa138031160 legacy=0x7fa138033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:48.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.308+0000 7fa151d9d640 1 -- 172.21.15.102:0/1631929607 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa14c128c40 msgr2=0x7fa14c1153c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:48.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.308+0000 7fa151d9d640 1 --2- 172.21.15.102:0/1631929607 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa14c128c40 0x7fa14c1153c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa14000ea20 tx=0x7fa1400090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.308+0000 7fa151d9d640 1 -- 172.21.15.102:0/1631929607 shutdown_connections 2024-09-13T17:43:48.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.308+0000 7fa151d9d640 1 --2- 172.21.15.102:0/1631929607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa14c12bd80 0x7fa14c115900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.308+0000 7fa151d9d640 1 --2- 172.21.15.102:0/1631929607 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa14c128c40 0x7fa14c1153c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.308+0000 7fa151d9d640 1 --2- 172.21.15.102:0/1631929607 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa14c128330 0x7fa14c114e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.308+0000 7fa151d9d640 1 -- 172.21.15.102:0/1631929607 >> 172.21.15.102:0/1631929607 conn(0x7fa14c09e820 msgr2=0x7fa14c1261a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:48.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.308+0000 7fa151d9d640 1 -- 172.21.15.102:0/1631929607 shutdown_connections 2024-09-13T17:43:48.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.308+0000 7fa151d9d640 1 -- 172.21.15.102:0/1631929607 wait complete. 2024-09-13T17:43:48.321 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:54.442612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T17:41:59.076782+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 16, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:42:04.672265+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:42:05.583134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:43:48.321 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:43:48.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.415+0000 7f5cb0280640 1 Processor -- start 2024-09-13T17:43:48.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.415+0000 7f5cb0280640 1 -- start start 2024-09-13T17:43:48.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.415+0000 7f5cb0280640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5ca8098010 0x7f5ca8130f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:48.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.415+0000 7f5cb0280640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5ca80983e0 0x7f5ca8133470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:48.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.415+0000 7f5cb0280640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ca8098c50 con 0x7f5ca8097530 2024-09-13T17:43:48.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.415+0000 7f5cb0280640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ca806ed00 con 0x7f5ca80983e0 2024-09-13T17:43:48.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.415+0000 7f5cb0280640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca8068bd0 con 0x7f5ca8098010 2024-09-13T17:43:48.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.415+0000 7f5cad81b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5ca8098010 0x7f5ca8130f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:48.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.415+0000 7f5cae01c640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f5ca8097530 0x7f5ca8097900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36422/0 (socket says 172.21.15.102:36422) 2024-09-13T17:43:48.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.415+0000 7f5cae01c640 1 -- 172.21.15.102:0/4259917048 learned_addr learned my addr 172.21.15.102:0/4259917048 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:48.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.415+0000 7f5cae81d640 1 --2- 172.21.15.102:0/4259917048 >> v2:172.21.15.102:3300/0 conn(0x7f5ca80983e0 0x7f5ca8133470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:48.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.416+0000 7f5cae81d640 1 -- 172.21.15.102:0/4259917048 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5ca8098010 msgr2=0x7f5ca8130f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:48.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.416+0000 7f5cae81d640 1 --2- 172.21.15.102:0/4259917048 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5ca8098010 0x7f5ca8130f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.416+0000 7f5cae81d640 1 -- 172.21.15.102:0/4259917048 >> v1:172.21.15.102:6789/0 conn(0x7f5ca8097530 legacy=0x7f5ca8097900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:48.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.416+0000 7f5cad01a640 1 -- 172.21.15.102:0/4259917048 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3749412784 0 0) 0x7f5ca8098c50 con 0x7f5ca8097530 2024-09-13T17:43:48.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.416+0000 7f5cae81d640 1 -- 172.21.15.102:0/4259917048 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca8099040 con 0x7f5ca80983e0 2024-09-13T17:43:48.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.416+0000 7f5cad81b640 1 --2- 172.21.15.102:0/4259917048 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5ca8098010 0x7f5ca8130f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:48.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.416+0000 7f5cae81d640 1 --2- 172.21.15.102:0/4259917048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ca80983e0 0x7f5ca8133470 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000af70 tx=0x7f5ca000a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e561834db4645a53 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:48.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.417+0000 7f5cad01a640 1 -- 172.21.15.102:0/4259917048 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca000a2e0 con 0x7f5ca80983e0 2024-09-13T17:43:48.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.417+0000 7f5cad01a640 1 -- 172.21.15.102:0/4259917048 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ca002e050 con 0x7f5ca80983e0 2024-09-13T17:43:48.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.417+0000 7f5cad01a640 1 -- 172.21.15.102:0/4259917048 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca0008680 con 0x7f5ca80983e0 2024-09-13T17:43:48.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.417+0000 7f5cb0280640 1 -- 172.21.15.102:0/4259917048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ca80983e0 msgr2=0x7f5ca8133470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:48.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.417+0000 7f5cb0280640 1 --2- 172.21.15.102:0/4259917048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ca80983e0 0x7f5ca8133470 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000af70 tx=0x7f5ca000a890 comp rx=0 tx=0).stop 2024-09-13T17:43:48.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.417+0000 7f5cb0280640 1 -- 172.21.15.102:0/4259917048 shutdown_connections 2024-09-13T17:43:48.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.417+0000 7f5cb0280640 1 --2- 172.21.15.102:0/4259917048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ca80983e0 0x7f5ca8133470 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.417+0000 7f5cb0280640 1 --2- 172.21.15.102:0/4259917048 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5ca8098010 0x7f5ca8130f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.417+0000 7f5cb0280640 1 -- 172.21.15.102:0/4259917048 >> 172.21.15.102:0/4259917048 conn(0x7f5ca809e720 msgr2=0x7f5ca809eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:48.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.417+0000 7f5cb0280640 1 -- 172.21.15.102:0/4259917048 shutdown_connections 2024-09-13T17:43:48.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.417+0000 7f5cb0280640 1 -- 172.21.15.102:0/4259917048 wait complete. 2024-09-13T17:43:48.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.418+0000 7f5cb0280640 1 Processor -- start 2024-09-13T17:43:48.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.418+0000 7f5cb0280640 1 -- start start 2024-09-13T17:43:48.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.418+0000 7f5cb0280640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ca8097530 0x7f5ca8119940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:48.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.418+0000 7f5cb0280640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ca8098010 0x7f5ca8119e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:48.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.418+0000 7f5cb0280640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5ca80983e0 0x7f5ca811a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:48.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.418+0000 7f5cb0280640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca806dfc0 con 0x7f5ca8097530 2024-09-13T17:43:48.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.418+0000 7f5cb0280640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca809df90 con 0x7f5ca8098010 2024-09-13T17:43:48.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.418+0000 7f5cb0280640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca811baa0 con 0x7f5ca80983e0 2024-09-13T17:43:48.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.419+0000 7f5cae01c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ca8097530 0x7f5ca8119940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:48.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.419+0000 7f5cad81b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ca8098010 0x7f5ca8119e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:48.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.419+0000 7f5cae01c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ca8097530 0x7f5ca8119940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57816/0 (socket says 172.21.15.102:57816) 2024-09-13T17:43:48.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.419+0000 7f5cae01c640 1 -- 172.21.15.102:0/1048980631 learned_addr learned my addr 172.21.15.102:0/1048980631 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:48.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.419+0000 7f5cae81d640 1 --2- 172.21.15.102:0/1048980631 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5ca80983e0 0x7f5ca811a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:48.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.419+0000 7f5cae01c640 1 -- 172.21.15.102:0/1048980631 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5ca80983e0 msgr2=0x7f5ca811a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:48.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.419+0000 7f5cae01c640 1 --2- 172.21.15.102:0/1048980631 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5ca80983e0 0x7f5ca811a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.419+0000 7f5cae01c640 1 -- 172.21.15.102:0/1048980631 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ca8098010 msgr2=0x7f5ca8119e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:48.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.419+0000 7f5cae01c640 1 --2- 172.21.15.102:0/1048980631 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ca8098010 0x7f5ca8119e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.419+0000 7f5cae01c640 1 -- 172.21.15.102:0/1048980631 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9c009390 con 0x7f5ca8097530 2024-09-13T17:43:48.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.419+0000 7f5cad81b640 1 --2- 172.21.15.102:0/1048980631 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ca8098010 0x7f5ca8119e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:48.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.419+0000 7f5cae01c640 1 --2- 172.21.15.102:0/1048980631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ca8097530 0x7f5ca8119940 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400c5e0 tx=0x7f5ca4007d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:48.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.419+0000 7f5c9a7fc640 1 -- 172.21.15.102:0/1048980631 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4005aa0 con 0x7f5ca8097530 2024-09-13T17:43:48.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.419+0000 7f5c9a7fc640 1 -- 172.21.15.102:0/1048980631 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ca4005c00 con 0x7f5ca8097530 2024-09-13T17:43:48.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.419+0000 7f5cb0280640 1 -- 172.21.15.102:0/1048980631 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ca000ab30 con 0x7f5ca8097530 2024-09-13T17:43:48.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.419+0000 7f5c9a7fc640 1 -- 172.21.15.102:0/1048980631 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca40070f0 con 0x7f5ca8097530 2024-09-13T17:43:48.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.419+0000 7f5cb0280640 1 -- 172.21.15.102:0/1048980631 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ca811aea0 con 0x7f5ca8097530 2024-09-13T17:43:48.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.420+0000 7f5c9a7fc640 1 -- 172.21.15.102:0/1048980631 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f5ca4007520 con 0x7f5ca8097530 2024-09-13T17:43:48.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.420+0000 7f5c9a7fc640 1 -- 172.21.15.102:0/1048980631 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f5ca4041bd0 con 0x7f5ca8097530 2024-09-13T17:43:48.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.420+0000 7f5cb0280640 1 -- 172.21.15.102:0/1048980631 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ca811cab0 con 0x7f5ca8097530 2024-09-13T17:43:48.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.424+0000 7f5c9a7fc640 1 -- 172.21.15.102:0/1048980631 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f5ca401b400 con 0x7f5ca8097530 2024-09-13T17:43:48.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.579+0000 7f5cb0280640 1 -- 172.21.15.102:0/1048980631 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5ca8064000 con 0x7f5ca8097530 2024-09-13T17:43:48.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.579+0000 7f5c9a7fc640 1 -- 172.21.15.102:0/1048980631 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5ca4019300 con 0x7f5ca8097530 2024-09-13T17:43:48.581 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-09-13T17:43:48.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.581+0000 7f5cb0280640 1 -- 172.21.15.102:0/1048980631 >> v1:172.21.15.102:6816/34937 conn(0x7f5c8c031160 legacy=0x7f5c8c033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:48.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.581+0000 7f5cb0280640 1 -- 172.21.15.102:0/1048980631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ca8097530 msgr2=0x7f5ca8119940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:48.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.581+0000 7f5cb0280640 1 --2- 172.21.15.102:0/1048980631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ca8097530 0x7f5ca8119940 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400c5e0 tx=0x7f5ca4007d00 comp rx=0 tx=0).stop 2024-09-13T17:43:48.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.581+0000 7f5cb0280640 1 -- 172.21.15.102:0/1048980631 shutdown_connections 2024-09-13T17:43:48.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.581+0000 7f5cb0280640 1 --2- 172.21.15.102:0/1048980631 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5ca80983e0 0x7f5ca811a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.581+0000 7f5cb0280640 1 --2- 172.21.15.102:0/1048980631 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5ca8098010 0x7f5ca8119e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.581+0000 7f5cb0280640 1 --2- 172.21.15.102:0/1048980631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ca8097530 0x7f5ca8119940 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.581+0000 7f5cb0280640 1 -- 172.21.15.102:0/1048980631 >> 172.21.15.102:0/1048980631 conn(0x7f5ca809e720 msgr2=0x7f5ca812df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:48.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.581+0000 7f5cb0280640 1 -- 172.21.15.102:0/1048980631 shutdown_connections 2024-09-13T17:43:48.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.581+0000 7f5cb0280640 1 -- 172.21.15.102:0/1048980631 wait complete. 2024-09-13T17:43:48.594 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-13T17:43:48.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.688+0000 7f2a4191d640 1 Processor -- start 2024-09-13T17:43:48.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.688+0000 7f2a4191d640 1 -- start start 2024-09-13T17:43:48.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.689+0000 7f2a4191d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2a3c09df40 0x7f2a3c134960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:48.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.689+0000 7f2a4191d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2a3c134ea0 0x7f2a3c137280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:48.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.689+0000 7f2a4191d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a3c096b10 con 0x7f2a3c0689d0 2024-09-13T17:43:48.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.689+0000 7f2a4191d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a3c068da0 con 0x7f2a3c09df40 2024-09-13T17:43:48.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.689+0000 7f2a4191d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c096db0 con 0x7f2a3c134ea0 2024-09-13T17:43:48.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.689+0000 7f2a3a7fc640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2a3c09df40 0x7f2a3c134960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:48.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.689+0000 7f2a3b7fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2a3c134ea0 0x7f2a3c137280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:48.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.689+0000 7f2a3affd640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f2a3c0689d0 0x7f2a3c09d830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36434/0 (socket says 172.21.15.102:36434) 2024-09-13T17:43:48.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.689+0000 7f2a3affd640 1 -- 172.21.15.102:0/729007322 learned_addr learned my addr 172.21.15.102:0/729007322 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:48.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.689+0000 7f2a3a7fc640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2a3c09df40 0x7f2a3c134960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57822/0 (socket says 172.21.15.102:57822) 2024-09-13T17:43:48.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.689+0000 7f2a3a7fc640 1 -- 172.21.15.102:0/729007322 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2a3c134ea0 msgr2=0x7f2a3c137280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:48.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.689+0000 7f2a3a7fc640 1 --2- 172.21.15.102:0/729007322 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2a3c134ea0 0x7f2a3c137280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.689+0000 7f2a3a7fc640 1 -- 172.21.15.102:0/729007322 >> v1:172.21.15.102:6789/0 conn(0x7f2a3c0689d0 legacy=0x7f2a3c09d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:48.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.689+0000 7f2a3a7fc640 1 -- 172.21.15.102:0/729007322 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3c0696e0 con 0x7f2a3c09df40 2024-09-13T17:43:48.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.689+0000 7f2a39ffb640 1 -- 172.21.15.102:0/729007322 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 500262510 0 0) 0x7f2a3c096b10 con 0x7f2a3c0689d0 2024-09-13T17:43:48.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.689+0000 7f2a3b7fe640 1 --2- 172.21.15.102:0/729007322 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2a3c134ea0 0x7f2a3c137280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:48.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.690+0000 7f2a3a7fc640 1 --2- 172.21.15.102:0/729007322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a3c09df40 0x7f2a3c134960 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2a30009850 tx=0x7f2a300090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=735a51ab8fe81ca4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:48.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.690+0000 7f2a39ffb640 1 -- 172.21.15.102:0/729007322 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3002ac80 con 0x7f2a3c09df40 2024-09-13T17:43:48.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.690+0000 7f2a39ffb640 1 -- 172.21.15.102:0/729007322 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a3002ade0 con 0x7f2a3c09df40 2024-09-13T17:43:48.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.690+0000 7f2a39ffb640 1 -- 172.21.15.102:0/729007322 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a30032670 con 0x7f2a3c09df40 2024-09-13T17:43:48.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.691+0000 7f2a4191d640 1 -- 172.21.15.102:0/729007322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a3c09df40 msgr2=0x7f2a3c134960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:48.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.691+0000 7f2a4191d640 1 --2- 172.21.15.102:0/729007322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a3c09df40 0x7f2a3c134960 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2a30009850 tx=0x7f2a300090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.691+0000 7f2a4191d640 1 -- 172.21.15.102:0/729007322 shutdown_connections 2024-09-13T17:43:48.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.691+0000 7f2a4191d640 1 --2- 172.21.15.102:0/729007322 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2a3c134ea0 0x7f2a3c137280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.691+0000 7f2a4191d640 1 --2- 172.21.15.102:0/729007322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a3c09df40 0x7f2a3c134960 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.691+0000 7f2a4191d640 1 -- 172.21.15.102:0/729007322 >> 172.21.15.102:0/729007322 conn(0x7f2a3c06f320 msgr2=0x7f2a3c0a2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:48.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.691+0000 7f2a4191d640 1 -- 172.21.15.102:0/729007322 shutdown_connections 2024-09-13T17:43:48.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.691+0000 7f2a4191d640 1 -- 172.21.15.102:0/729007322 wait complete. 2024-09-13T17:43:48.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.692+0000 7f2a4191d640 1 Processor -- start 2024-09-13T17:43:48.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.692+0000 7f2a4191d640 1 -- start start 2024-09-13T17:43:48.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.692+0000 7f2a4191d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a3c0689d0 0x7f2a3c11e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:48.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.692+0000 7f2a4191d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2a3c09df40 0x7f2a3c11a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:48.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.692+0000 7f2a4191d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2a3c134ea0 0x7f2a3c11a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:48.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.692+0000 7f2a4191d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c11c090 con 0x7f2a3c0689d0 2024-09-13T17:43:48.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.692+0000 7f2a4191d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c11d470 con 0x7f2a3c134ea0 2024-09-13T17:43:48.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.692+0000 7f2a4191d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c130b40 con 0x7f2a3c09df40 2024-09-13T17:43:48.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.692+0000 7f2a3affd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a3c0689d0 0x7f2a3c11e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:48.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.692+0000 7f2a3a7fc640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2a3c09df40 0x7f2a3c11a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:48.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.692+0000 7f2a3affd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a3c0689d0 0x7f2a3c11e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57830/0 (socket says 172.21.15.102:57830) 2024-09-13T17:43:48.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.692+0000 7f2a3affd640 1 -- 172.21.15.102:0/3434855312 learned_addr learned my addr 172.21.15.102:0/3434855312 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:48.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.692+0000 7f2a3b7fe640 1 --2- 172.21.15.102:0/3434855312 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2a3c134ea0 0x7f2a3c11a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:48.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.693+0000 7f2a3affd640 1 -- 172.21.15.102:0/3434855312 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2a3c09df40 msgr2=0x7f2a3c11a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:48.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.693+0000 7f2a3affd640 1 --2- 172.21.15.102:0/3434855312 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2a3c09df40 0x7f2a3c11a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.693+0000 7f2a3affd640 1 -- 172.21.15.102:0/3434855312 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2a3c134ea0 msgr2=0x7f2a3c11a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:48.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.693+0000 7f2a3affd640 1 --2- 172.21.15.102:0/3434855312 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2a3c134ea0 0x7f2a3c11a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.693+0000 7f2a3affd640 1 -- 172.21.15.102:0/3434855312 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a34009a20 con 0x7f2a3c0689d0 2024-09-13T17:43:48.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.693+0000 7f2a3a7fc640 1 --2- 172.21.15.102:0/3434855312 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2a3c09df40 0x7f2a3c11a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:48.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.693+0000 7f2a3affd640 1 --2- 172.21.15.102:0/3434855312 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a3c0689d0 0x7f2a3c11e2b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c00c510 tx=0x7f2a2c007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:48.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.693+0000 7f2a22ffd640 1 -- 172.21.15.102:0/3434855312 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c005aa0 con 0x7f2a3c0689d0 2024-09-13T17:43:48.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.693+0000 7f2a22ffd640 1 -- 172.21.15.102:0/3434855312 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2c005c00 con 0x7f2a3c0689d0 2024-09-13T17:43:48.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.693+0000 7f2a4191d640 1 -- 172.21.15.102:0/3434855312 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a30009390 con 0x7f2a3c0689d0 2024-09-13T17:43:48.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.693+0000 7f2a22ffd640 1 -- 172.21.15.102:0/3434855312 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c0070d0 con 0x7f2a3c0689d0 2024-09-13T17:43:48.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.693+0000 7f2a4191d640 1 -- 172.21.15.102:0/3434855312 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a3c11d810 con 0x7f2a3c0689d0 2024-09-13T17:43:48.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.694+0000 7f2a22ffd640 1 -- 172.21.15.102:0/3434855312 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f2a2c0074c0 con 0x7f2a3c0689d0 2024-09-13T17:43:48.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.694+0000 7f2a22ffd640 1 -- 172.21.15.102:0/3434855312 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f2a2c041540 con 0x7f2a3c0689d0 2024-09-13T17:43:48.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.694+0000 7f2a4191d640 1 -- 172.21.15.102:0/3434855312 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a3c064000 con 0x7f2a3c0689d0 2024-09-13T17:43:48.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.698+0000 7f2a22ffd640 1 -- 172.21.15.102:0/3434855312 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f2a2c017070 con 0x7f2a3c0689d0 2024-09-13T17:43:48.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.850+0000 7f2a4191d640 1 -- 172.21.15.102:0/3434855312 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2a3c0a39c0 con 0x7f2a3c0689d0 2024-09-13T17:43:48.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.851+0000 7f2a22ffd640 1 -- 172.21.15.102:0/3434855312 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2a2c01a360 con 0x7f2a3c0689d0 2024-09-13T17:43:48.853 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-09-13T17:43:48.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.853+0000 7f2a4191d640 1 -- 172.21.15.102:0/3434855312 >> v1:172.21.15.102:6816/34937 conn(0x7f2a28031160 legacy=0x7f2a28033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:48.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.853+0000 7f2a4191d640 1 -- 172.21.15.102:0/3434855312 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a3c0689d0 msgr2=0x7f2a3c11e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:48.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.853+0000 7f2a4191d640 1 --2- 172.21.15.102:0/3434855312 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a3c0689d0 0x7f2a3c11e2b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c00c510 tx=0x7f2a2c007f30 comp rx=0 tx=0).stop 2024-09-13T17:43:48.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.853+0000 7f2a4191d640 1 -- 172.21.15.102:0/3434855312 shutdown_connections 2024-09-13T17:43:48.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.853+0000 7f2a4191d640 1 --2- 172.21.15.102:0/3434855312 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2a3c134ea0 0x7f2a3c11a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.853+0000 7f2a4191d640 1 --2- 172.21.15.102:0/3434855312 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2a3c09df40 0x7f2a3c11a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.853+0000 7f2a4191d640 1 --2- 172.21.15.102:0/3434855312 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a3c0689d0 0x7f2a3c11e2b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.853+0000 7f2a4191d640 1 -- 172.21.15.102:0/3434855312 >> 172.21.15.102:0/3434855312 conn(0x7f2a3c06f320 msgr2=0x7f2a3c0a18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:48.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.853+0000 7f2a4191d640 1 -- 172.21.15.102:0/3434855312 shutdown_connections 2024-09-13T17:43:48.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.853+0000 7f2a4191d640 1 -- 172.21.15.102:0/3434855312 wait complete. 2024-09-13T17:43:48.865 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:43:48.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.959+0000 7eff4339d640 1 Processor -- start 2024-09-13T17:43:48.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.959+0000 7eff4339d640 1 -- start start 2024-09-13T17:43:48.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.960+0000 7eff4339d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7eff3c096a70 0x7eff3c12c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:48.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.960+0000 7eff4339d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7eff3c12c900 0x7eff3c098410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:48.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.960+0000 7eff4339d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff3c099540 con 0x7eff3c12ccd0 2024-09-13T17:43:48.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.960+0000 7eff4339d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff3c096e40 con 0x7eff3c12c900 2024-09-13T17:43:48.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.960+0000 7eff4339d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c09a8e0 con 0x7eff3c096a70 2024-09-13T17:43:48.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.960+0000 7eff40938640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7eff3c12c900 0x7eff3c098410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:48.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.960+0000 7eff41139640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7eff3c096a70 0x7eff3c12c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:48.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.960+0000 7eff4193a640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7eff3c12ccd0 0x7eff3c098950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36444/0 (socket says 172.21.15.102:36444) 2024-09-13T17:43:48.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.960+0000 7eff40938640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7eff3c12c900 0x7eff3c098410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57832/0 (socket says 172.21.15.102:57832) 2024-09-13T17:43:48.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.960+0000 7eff4193a640 1 -- 172.21.15.102:0/4181474255 learned_addr learned my addr 172.21.15.102:0/4181474255 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:48.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.960+0000 7eff40938640 1 -- 172.21.15.102:0/4181474255 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7eff3c096a70 msgr2=0x7eff3c12c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:48.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.960+0000 7eff40938640 1 --2- 172.21.15.102:0/4181474255 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7eff3c096a70 0x7eff3c12c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.960+0000 7eff40938640 1 -- 172.21.15.102:0/4181474255 >> v1:172.21.15.102:6789/0 conn(0x7eff3c12ccd0 legacy=0x7eff3c098950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:48.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.960+0000 7eff40938640 1 -- 172.21.15.102:0/4181474255 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff3c099760 con 0x7eff3c12c900 2024-09-13T17:43:48.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.960+0000 7eff3bfff640 1 -- 172.21.15.102:0/4181474255 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1040277845 0 0) 0x7eff3c099540 con 0x7eff3c12ccd0 2024-09-13T17:43:48.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.960+0000 7eff41139640 1 --2- 172.21.15.102:0/4181474255 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7eff3c096a70 0x7eff3c12c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:48.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.961+0000 7eff40938640 1 --2- 172.21.15.102:0/4181474255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff3c12c900 0x7eff3c098410 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7eff30009850 tx=0x7eff300090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d597597c30d402 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:48.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.961+0000 7eff3bfff640 1 -- 172.21.15.102:0/4181474255 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3002ad00 con 0x7eff3c12c900 2024-09-13T17:43:48.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.961+0000 7eff3bfff640 1 -- 172.21.15.102:0/4181474255 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff3002ae60 con 0x7eff3c12c900 2024-09-13T17:43:48.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.961+0000 7eff3bfff640 1 -- 172.21.15.102:0/4181474255 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff30032670 con 0x7eff3c12c900 2024-09-13T17:43:48.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.961+0000 7eff4339d640 1 -- 172.21.15.102:0/4181474255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff3c12c900 msgr2=0x7eff3c098410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:48.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.961+0000 7eff4339d640 1 --2- 172.21.15.102:0/4181474255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff3c12c900 0x7eff3c098410 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7eff30009850 tx=0x7eff300090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.962+0000 7eff4339d640 1 -- 172.21.15.102:0/4181474255 shutdown_connections 2024-09-13T17:43:48.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.962+0000 7eff4339d640 1 --2- 172.21.15.102:0/4181474255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff3c12c900 0x7eff3c098410 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.962+0000 7eff4339d640 1 --2- 172.21.15.102:0/4181474255 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7eff3c096a70 0x7eff3c12c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.962+0000 7eff4339d640 1 -- 172.21.15.102:0/4181474255 >> 172.21.15.102:0/4181474255 conn(0x7eff3c09e6b0 msgr2=0x7eff3c09eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:48.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.962+0000 7eff4339d640 1 -- 172.21.15.102:0/4181474255 shutdown_connections 2024-09-13T17:43:48.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.962+0000 7eff4339d640 1 -- 172.21.15.102:0/4181474255 wait complete. 2024-09-13T17:43:48.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.962+0000 7eff4339d640 1 Processor -- start 2024-09-13T17:43:48.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.962+0000 7eff4339d640 1 -- start start 2024-09-13T17:43:48.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.963+0000 7eff4339d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7eff3c096a70 0x7eff3c12f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:48.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.963+0000 7eff4339d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff3c12c900 0x7eff3c12fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:48.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.963+0000 7eff4339d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7eff3c12ccd0 0x7eff3c130350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:48.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.963+0000 7eff4339d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c1318f0 con 0x7eff3c12c900 2024-09-13T17:43:48.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.963+0000 7eff4339d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c132fe0 con 0x7eff3c12ccd0 2024-09-13T17:43:48.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.963+0000 7eff4339d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c131450 con 0x7eff3c096a70 2024-09-13T17:43:48.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.963+0000 7eff41139640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7eff3c096a70 0x7eff3c12f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:48.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.963+0000 7eff40938640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff3c12c900 0x7eff3c12fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:48.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.963+0000 7eff41139640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7eff3c096a70 0x7eff3c12f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:43480/0 (socket says 172.21.15.102:43480) 2024-09-13T17:43:48.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.963+0000 7eff41139640 1 -- 172.21.15.102:0/1454040231 learned_addr learned my addr 172.21.15.102:0/1454040231 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:48.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.963+0000 7eff4193a640 1 --2- 172.21.15.102:0/1454040231 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7eff3c12ccd0 0x7eff3c130350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:48.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.963+0000 7eff40938640 1 -- 172.21.15.102:0/1454040231 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7eff3c096a70 msgr2=0x7eff3c12f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:48.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.963+0000 7eff40938640 1 --2- 172.21.15.102:0/1454040231 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7eff3c096a70 0x7eff3c12f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.963+0000 7eff40938640 1 -- 172.21.15.102:0/1454040231 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7eff3c12ccd0 msgr2=0x7eff3c130350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:48.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.963+0000 7eff40938640 1 --2- 172.21.15.102:0/1454040231 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7eff3c12ccd0 0x7eff3c130350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.963+0000 7eff40938640 1 -- 172.21.15.102:0/1454040231 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff2c009a20 con 0x7eff3c12c900 2024-09-13T17:43:48.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.963+0000 7eff41139640 1 --2- 172.21.15.102:0/1454040231 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7eff3c096a70 0x7eff3c12f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:48.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.964+0000 7eff40938640 1 --2- 172.21.15.102:0/1454040231 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff3c12c900 0x7eff3c12fe10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7eff3c09a410 tx=0x7eff30032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:48.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.964+0000 7eff397fa640 1 -- 172.21.15.102:0/1454040231 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff30031700 con 0x7eff3c12c900 2024-09-13T17:43:48.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.964+0000 7eff397fa640 1 -- 172.21.15.102:0/1454040231 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff30031880 con 0x7eff3c12c900 2024-09-13T17:43:48.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.964+0000 7eff4339d640 1 -- 172.21.15.102:0/1454040231 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff30009390 con 0x7eff3c12c900 2024-09-13T17:43:48.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.964+0000 7eff397fa640 1 -- 172.21.15.102:0/1454040231 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff30031c40 con 0x7eff3c12c900 2024-09-13T17:43:48.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.964+0000 7eff4339d640 1 -- 172.21.15.102:0/1454040231 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff3c130cd0 con 0x7eff3c12c900 2024-09-13T17:43:48.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.965+0000 7eff397fa640 1 -- 172.21.15.102:0/1454040231 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7eff30039040 con 0x7eff3c12c900 2024-09-13T17:43:48.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.965+0000 7eff397fa640 1 -- 172.21.15.102:0/1454040231 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7eff30061b70 con 0x7eff3c12c900 2024-09-13T17:43:48.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.965+0000 7eff4339d640 1 -- 172.21.15.102:0/1454040231 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff3c064000 con 0x7eff3c12c900 2024-09-13T17:43:48.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:48.969+0000 7eff397fa640 1 -- 172.21.15.102:0/1454040231 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7eff3003b610 con 0x7eff3c12c900 2024-09-13T17:43:49.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.118+0000 7eff4339d640 1 -- 172.21.15.102:0/1454040231 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7eff3c133f00 con 0x7eff3c12c900 2024-09-13T17:43:49.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.119+0000 7eff397fa640 1 -- 172.21.15.102:0/1454040231 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7eff3003f070 con 0x7eff3c12c900 2024-09-13T17:43:49.121 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-09-13T17:43:49.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.121+0000 7eff4339d640 1 -- 172.21.15.102:0/1454040231 >> v1:172.21.15.102:6816/34937 conn(0x7eff28031160 legacy=0x7eff28033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:49.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.121+0000 7eff4339d640 1 -- 172.21.15.102:0/1454040231 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff3c12c900 msgr2=0x7eff3c12fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:49.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.121+0000 7eff4339d640 1 --2- 172.21.15.102:0/1454040231 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff3c12c900 0x7eff3c12fe10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7eff3c09a410 tx=0x7eff30032000 comp rx=0 tx=0).stop 2024-09-13T17:43:49.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.121+0000 7eff4339d640 1 -- 172.21.15.102:0/1454040231 shutdown_connections 2024-09-13T17:43:49.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.121+0000 7eff4339d640 1 --2- 172.21.15.102:0/1454040231 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7eff3c12ccd0 0x7eff3c130350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:49.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.121+0000 7eff4339d640 1 --2- 172.21.15.102:0/1454040231 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff3c12c900 0x7eff3c12fe10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:49.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.121+0000 7eff4339d640 1 --2- 172.21.15.102:0/1454040231 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7eff3c096a70 0x7eff3c12f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:49.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.121+0000 7eff4339d640 1 -- 172.21.15.102:0/1454040231 >> 172.21.15.102:0/1454040231 conn(0x7eff3c09e6b0 msgr2=0x7eff3c097f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:49.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.121+0000 7eff4339d640 1 -- 172.21.15.102:0/1454040231 shutdown_connections 2024-09-13T17:43:49.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.121+0000 7eff4339d640 1 -- 172.21.15.102:0/1454040231 wait complete. 2024-09-13T17:43:49.133 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:43:49.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.227+0000 7fb78799d640 1 Processor -- start 2024-09-13T17:43:49.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.227+0000 7fb78799d640 1 -- start start 2024-09-13T17:43:49.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.228+0000 7fb78799d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb780098ed0 0x7fb780131610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:49.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.228+0000 7fb78799d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb780131b50 0x7fb780133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:49.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.228+0000 7fb78799d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7800999e0 con 0x7fb780096a70 2024-09-13T17:43:49.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.228+0000 7fb78799d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb780069c30 con 0x7fb780131b50 2024-09-13T17:43:49.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.228+0000 7fb78799d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb78006ed00 con 0x7fb780098ed0 2024-09-13T17:43:49.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.228+0000 7fb785f3a640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb780131b50 0x7fb780133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:49.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.228+0000 7fb784f38640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb780098ed0 0x7fb780131610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:49.230 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.228+0000 7fb785739640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fb780096a70 0x7fb7800987c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36458/0 (socket says 172.21.15.102:36458) 2024-09-13T17:43:49.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.228+0000 7fb785739640 1 -- 172.21.15.102:0/2204705136 learned_addr learned my addr 172.21.15.102:0/2204705136 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:49.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.228+0000 7fb785f3a640 1 -- 172.21.15.102:0/2204705136 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb780098ed0 msgr2=0x7fb780131610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:49.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.228+0000 7fb785f3a640 1 --2- 172.21.15.102:0/2204705136 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb780098ed0 0x7fb780131610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:49.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.228+0000 7fb785f3a640 1 -- 172.21.15.102:0/2204705136 >> v1:172.21.15.102:6789/0 conn(0x7fb780096a70 legacy=0x7fb7800987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:49.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.228+0000 7fb785f3a640 1 -- 172.21.15.102:0/2204705136 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7800997a0 con 0x7fb780131b50 2024-09-13T17:43:49.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.228+0000 7fb77ffff640 1 -- 172.21.15.102:0/2204705136 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 662349199 0 0) 0x7fb7800999e0 con 0x7fb780096a70 2024-09-13T17:43:49.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.228+0000 7fb784f38640 1 --2- 172.21.15.102:0/2204705136 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb780098ed0 0x7fb780131610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:49.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.229+0000 7fb785f3a640 1 --2- 172.21.15.102:0/2204705136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb780131b50 0x7fb780133f30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb770009880 tx=0x7fb7700090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c95bdf3637eef0c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:49.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.229+0000 7fb77ffff640 1 -- 172.21.15.102:0/2204705136 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77002ace0 con 0x7fb780131b50 2024-09-13T17:43:49.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.229+0000 7fb77ffff640 1 -- 172.21.15.102:0/2204705136 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77002ae40 con 0x7fb780131b50 2024-09-13T17:43:49.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.229+0000 7fb77ffff640 1 -- 172.21.15.102:0/2204705136 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb770032670 con 0x7fb780131b50 2024-09-13T17:43:49.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.229+0000 7fb78799d640 1 -- 172.21.15.102:0/2204705136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb780131b50 msgr2=0x7fb780133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:49.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.229+0000 7fb78799d640 1 --2- 172.21.15.102:0/2204705136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb780131b50 0x7fb780133f30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb770009880 tx=0x7fb7700090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:49.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.229+0000 7fb78799d640 1 -- 172.21.15.102:0/2204705136 shutdown_connections 2024-09-13T17:43:49.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.229+0000 7fb78799d640 1 --2- 172.21.15.102:0/2204705136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb780131b50 0x7fb780133f30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:49.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.229+0000 7fb78799d640 1 --2- 172.21.15.102:0/2204705136 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb780098ed0 0x7fb780131610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:49.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.229+0000 7fb78799d640 1 -- 172.21.15.102:0/2204705136 >> 172.21.15.102:0/2204705136 conn(0x7fb78009e720 msgr2=0x7fb78009eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:49.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.230+0000 7fb78799d640 1 -- 172.21.15.102:0/2204705136 shutdown_connections 2024-09-13T17:43:49.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.230+0000 7fb78799d640 1 -- 172.21.15.102:0/2204705136 wait complete. 2024-09-13T17:43:49.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.230+0000 7fb78799d640 1 Processor -- start 2024-09-13T17:43:49.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.230+0000 7fb78799d640 1 -- start start 2024-09-13T17:43:49.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.231+0000 7fb78799d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb780096a70 0x7fb78006caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:49.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.231+0000 7fb78799d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb780098ed0 0x7fb78006cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:49.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.231+0000 7fb78799d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb780131b50 0x7fb78013a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:49.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.231+0000 7fb78799d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78009a410 con 0x7fb780131b50 2024-09-13T17:43:49.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.231+0000 7fb78799d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb78006e210 con 0x7fb780098ed0 2024-09-13T17:43:49.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.231+0000 7fb78799d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb780115ba0 con 0x7fb780096a70 2024-09-13T17:43:49.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.231+0000 7fb785f3a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb780131b50 0x7fb78013a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:49.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.231+0000 7fb785739640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb780096a70 0x7fb78006caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:49.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.231+0000 7fb784f38640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb780098ed0 0x7fb78006cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:49.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.231+0000 7fb785f3a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb780131b50 0x7fb78013a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57852/0 (socket says 172.21.15.102:57852) 2024-09-13T17:43:49.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.231+0000 7fb785f3a640 1 -- 172.21.15.102:0/784923711 learned_addr learned my addr 172.21.15.102:0/784923711 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:49.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.231+0000 7fb785f3a640 1 -- 172.21.15.102:0/784923711 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb780096a70 msgr2=0x7fb78006caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:49.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.231+0000 7fb785f3a640 1 --2- 172.21.15.102:0/784923711 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb780096a70 0x7fb78006caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:49.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.231+0000 7fb785f3a640 1 -- 172.21.15.102:0/784923711 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb780098ed0 msgr2=0x7fb78006cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:49.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.231+0000 7fb785f3a640 1 --2- 172.21.15.102:0/784923711 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb780098ed0 0x7fb78006cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:49.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.231+0000 7fb785f3a640 1 -- 172.21.15.102:0/784923711 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb774009390 con 0x7fb780131b50 2024-09-13T17:43:49.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.231+0000 7fb785739640 1 --2- 172.21.15.102:0/784923711 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb780096a70 0x7fb78006caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:49.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.231+0000 7fb784f38640 1 --2- 172.21.15.102:0/784923711 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb780098ed0 0x7fb78006cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:49.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.231+0000 7fb785f3a640 1 --2- 172.21.15.102:0/784923711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb780131b50 0x7fb78013a180 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb7700097d0 tx=0x7fb770032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:49.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.232+0000 7fb77d7fa640 1 -- 172.21.15.102:0/784923711 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77002ace0 con 0x7fb780131b50 2024-09-13T17:43:49.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.232+0000 7fb77d7fa640 1 -- 172.21.15.102:0/784923711 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb770031a60 con 0x7fb780131b50 2024-09-13T17:43:49.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.232+0000 7fb78799d640 1 -- 172.21.15.102:0/784923711 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb770009390 con 0x7fb780131b50 2024-09-13T17:43:49.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.232+0000 7fb77d7fa640 1 -- 172.21.15.102:0/784923711 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb770031e00 con 0x7fb780131b50 2024-09-13T17:43:49.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.232+0000 7fb78799d640 1 -- 172.21.15.102:0/784923711 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb780116890 con 0x7fb780131b50 2024-09-13T17:43:49.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.232+0000 7fb77d7fa640 1 -- 172.21.15.102:0/784923711 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fb770039040 con 0x7fb780131b50 2024-09-13T17:43:49.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.232+0000 7fb77d7fa640 1 -- 172.21.15.102:0/784923711 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7fb770041030 con 0x7fb780131b50 2024-09-13T17:43:49.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.233+0000 7fb78799d640 1 -- 172.21.15.102:0/784923711 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb780115d60 con 0x7fb780131b50 2024-09-13T17:43:49.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.237+0000 7fb77d7fa640 1 -- 172.21.15.102:0/784923711 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fb77004b020 con 0x7fb780131b50 2024-09-13T17:43:49.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.388+0000 7fb78799d640 1 -- 172.21.15.102:0/784923711 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb780064000 con 0x7fb780131b50 2024-09-13T17:43:49.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.388+0000 7fb77d7fa640 1 -- 172.21.15.102:0/784923711 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb770046480 con 0x7fb780131b50 2024-09-13T17:43:49.390 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-09-13T17:43:49.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.390+0000 7fb78799d640 1 -- 172.21.15.102:0/784923711 >> v1:172.21.15.102:6816/34937 conn(0x7fb76c031160 legacy=0x7fb76c033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:49.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.390+0000 7fb78799d640 1 -- 172.21.15.102:0/784923711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb780131b50 msgr2=0x7fb78013a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:49.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.390+0000 7fb78799d640 1 --2- 172.21.15.102:0/784923711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb780131b50 0x7fb78013a180 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb7700097d0 tx=0x7fb770032000 comp rx=0 tx=0).stop 2024-09-13T17:43:49.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.390+0000 7fb78799d640 1 -- 172.21.15.102:0/784923711 shutdown_connections 2024-09-13T17:43:49.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.390+0000 7fb78799d640 1 --2- 172.21.15.102:0/784923711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb780131b50 0x7fb78013a180 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:49.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.390+0000 7fb78799d640 1 --2- 172.21.15.102:0/784923711 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb780098ed0 0x7fb78006cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:49.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.390+0000 7fb78799d640 1 --2- 172.21.15.102:0/784923711 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb780096a70 0x7fb78006caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:49.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.390+0000 7fb78799d640 1 -- 172.21.15.102:0/784923711 >> 172.21.15.102:0/784923711 conn(0x7fb78009e720 msgr2=0x7fb78009d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:49.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.390+0000 7fb78799d640 1 -- 172.21.15.102:0/784923711 shutdown_connections 2024-09-13T17:43:49.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.390+0000 7fb78799d640 1 -- 172.21.15.102:0/784923711 wait complete. 2024-09-13T17:43:49.402 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_newops.TestNewOps.test_newops_getvxattr' 2024-09-13T17:43:49.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.494+0000 7f6d7c2c9640 1 Processor -- start 2024-09-13T17:43:49.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.494+0000 7f6d7c2c9640 1 -- start start 2024-09-13T17:43:49.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.495+0000 7f6d7c2c9640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6d74128330 0x7f6d74128700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:49.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.495+0000 7f6d7c2c9640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d74128c40 0x7f6d7412b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:49.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.495+0000 7f6d7c2c9640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d74097be0 con 0x7f6d7412bd80 2024-09-13T17:43:49.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.495+0000 7f6d7c2c9640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d74098e70 con 0x7f6d74128330 2024-09-13T17:43:49.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.495+0000 7f6d7c2c9640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d74098310 con 0x7f6d74128c40 2024-09-13T17:43:49.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.495+0000 7f6d79864640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d74128c40 0x7f6d7412b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:49.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.495+0000 7f6d7a065640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6d74128330 0x7f6d74128700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:49.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.495+0000 7f6d7a866640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f6d7412bd80 0x7f6d74132b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36474/0 (socket says 172.21.15.102:36474) 2024-09-13T17:43:49.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.495+0000 7f6d7a866640 1 -- 172.21.15.102:0/1696339883 learned_addr learned my addr 172.21.15.102:0/1696339883 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:49.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.496+0000 7f6d7a065640 1 -- 172.21.15.102:0/1696339883 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d74128c40 msgr2=0x7f6d7412b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:49.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.496+0000 7f6d7a065640 1 --2- 172.21.15.102:0/1696339883 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d74128c40 0x7f6d7412b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:49.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.496+0000 7f6d7a065640 1 -- 172.21.15.102:0/1696339883 >> v1:172.21.15.102:6789/0 conn(0x7f6d7412bd80 legacy=0x7f6d74132b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:49.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.496+0000 7f6d79063640 1 -- 172.21.15.102:0/1696339883 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3783696029 0 0) 0x7f6d74097be0 con 0x7f6d7412bd80 2024-09-13T17:43:49.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.496+0000 7f6d7a065640 1 -- 172.21.15.102:0/1696339883 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d740985d0 con 0x7f6d74128330 2024-09-13T17:43:49.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.496+0000 7f6d79864640 1 --2- 172.21.15.102:0/1696339883 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d74128c40 0x7f6d7412b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:49.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.496+0000 7f6d7a065640 1 --2- 172.21.15.102:0/1696339883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d74128330 0x7f6d74128700 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009850 tx=0x7f6d6c0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7619bd014f2a2852 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:49.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.496+0000 7f6d79063640 1 -- 172.21.15.102:0/1696339883 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c02ad90 con 0x7f6d74128330 2024-09-13T17:43:49.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.496+0000 7f6d79063640 1 -- 172.21.15.102:0/1696339883 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d6c032440 con 0x7f6d74128330 2024-09-13T17:43:49.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.496+0000 7f6d79063640 1 -- 172.21.15.102:0/1696339883 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c0327e0 con 0x7f6d74128330 2024-09-13T17:43:49.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.497+0000 7f6d7c2c9640 1 -- 172.21.15.102:0/1696339883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d74128330 msgr2=0x7f6d74128700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:49.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.497+0000 7f6d7c2c9640 1 --2- 172.21.15.102:0/1696339883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d74128330 0x7f6d74128700 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009850 tx=0x7f6d6c0090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:49.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.497+0000 7f6d7c2c9640 1 -- 172.21.15.102:0/1696339883 shutdown_connections 2024-09-13T17:43:49.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.497+0000 7f6d7c2c9640 1 --2- 172.21.15.102:0/1696339883 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d74128c40 0x7f6d7412b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:49.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.497+0000 7f6d7c2c9640 1 --2- 172.21.15.102:0/1696339883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d74128330 0x7f6d74128700 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:49.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.497+0000 7f6d7c2c9640 1 -- 172.21.15.102:0/1696339883 >> 172.21.15.102:0/1696339883 conn(0x7f6d7409e820 msgr2=0x7f6d7409ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:49.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.497+0000 7f6d7c2c9640 1 -- 172.21.15.102:0/1696339883 shutdown_connections 2024-09-13T17:43:49.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.497+0000 7f6d7c2c9640 1 -- 172.21.15.102:0/1696339883 wait complete. 2024-09-13T17:43:49.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.498+0000 7f6d7c2c9640 1 Processor -- start 2024-09-13T17:43:49.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.498+0000 7f6d7c2c9640 1 -- start start 2024-09-13T17:43:49.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.498+0000 7f6d7c2c9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6d74128330 0x7f6d7413aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:49.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.498+0000 7f6d7c2c9640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d74128c40 0x7f6d7413af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:49.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.498+0000 7f6d7c2c9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d7412bd80 0x7f6d7413b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:49.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.498+0000 7f6d7c2c9640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d741171d0 con 0x7f6d7412bd80 2024-09-13T17:43:49.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.498+0000 7f6d7c2c9640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7406eb70 con 0x7f6d74128c40 2024-09-13T17:43:49.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.498+0000 7f6d7c2c9640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6d74118290 con 0x7f6d74128330 2024-09-13T17:43:49.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.498+0000 7f6d7a065640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6d74128330 0x7f6d7413aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:49.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.498+0000 7f6d79864640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d74128c40 0x7f6d7413af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:49.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.498+0000 7f6d7a866640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d7412bd80 0x7f6d7413b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:49.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.498+0000 7f6d79864640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d74128c40 0x7f6d7413af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:51026/0 (socket says 172.21.15.102:51026) 2024-09-13T17:43:49.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.498+0000 7f6d79864640 1 -- 172.21.15.102:0/3247897518 learned_addr learned my addr 172.21.15.102:0/3247897518 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:49.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.498+0000 7f6d7a866640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d7412bd80 0x7f6d7413b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57870/0 (socket says 172.21.15.102:57870) 2024-09-13T17:43:49.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.499+0000 7f6d7a065640 1 -- 172.21.15.102:0/3247897518 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d74128c40 msgr2=0x7f6d7413af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:49.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.499+0000 7f6d7a065640 1 --2- 172.21.15.102:0/3247897518 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d74128c40 0x7f6d7413af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:49.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.499+0000 7f6d7a065640 1 -- 172.21.15.102:0/3247897518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d7412bd80 msgr2=0x7f6d7413b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:49.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.499+0000 7f6d7a065640 1 --2- 172.21.15.102:0/3247897518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d7412bd80 0x7f6d7413b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:49.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.499+0000 7f6d7a065640 1 -- 172.21.15.102:0/3247897518 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d68009390 con 0x7f6d74128330 2024-09-13T17:43:49.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.499+0000 7f6d7a866640 1 --2- 172.21.15.102:0/3247897518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d7412bd80 0x7f6d7413b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:49.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.499+0000 7f6d79864640 1 --2- 172.21.15.102:0/3247897518 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d74128c40 0x7f6d7413af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:49.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.499+0000 7f6d7a065640 1 --2- 172.21.15.102:0/3247897518 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6d74128330 0x7f6d7413aa20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c031000 tx=0x7f6d6c032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:49.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.499+0000 7f6d727fc640 1 -- 172.21.15.102:0/3247897518 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c000d10 con 0x7f6d74128330 2024-09-13T17:43:49.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.499+0000 7f6d727fc640 1 -- 172.21.15.102:0/3247897518 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d6c031a60 con 0x7f6d74128330 2024-09-13T17:43:49.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.499+0000 7f6d7c2c9640 1 -- 172.21.15.102:0/3247897518 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d6c009390 con 0x7f6d74128330 2024-09-13T17:43:49.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.499+0000 7f6d727fc640 1 -- 172.21.15.102:0/3247897518 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c031e20 con 0x7f6d74128330 2024-09-13T17:43:49.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.499+0000 7f6d7c2c9640 1 -- 172.21.15.102:0/3247897518 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d74125f40 con 0x7f6d74128330 2024-09-13T17:43:49.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.500+0000 7f6d727fc640 1 -- 172.21.15.102:0/3247897518 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f6d6c03e660 con 0x7f6d74128330 2024-09-13T17:43:49.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.500+0000 7f6d727fc640 1 -- 172.21.15.102:0/3247897518 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f6d6c0624c0 con 0x7f6d74128330 2024-09-13T17:43:49.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.500+0000 7f6d7c2c9640 1 -- 172.21.15.102:0/3247897518 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d74064000 con 0x7f6d74128330 2024-09-13T17:43:49.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.505+0000 7f6d727fc640 1 -- 172.21.15.102:0/3247897518 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f6d6c037070 con 0x7f6d74128330 2024-09-13T17:43:49.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:49.717+0000 7f6d7c2c9640 1 -- 172.21.15.102:0/3247897518 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_newops.TestNewOps.test_newops_getvxattr"]} v 0) v1 -- 0x7f6d7409a4e0 con 0x7f6d74128330 2024-09-13T17:43:50.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.542+0000 7f6d727fc640 1 -- 172.21.15.102:0/3247897518 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_newops.TestNewOps.test_newops_getvxattr"]}]=0 v96) v1 ==== 139+0+0 (secure 0 0 0) 0x7f6d6c03ee30 con 0x7f6d74128330 2024-09-13T17:43:50.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.544+0000 7f6d7c2c9640 1 -- 172.21.15.102:0/3247897518 >> v1:172.21.15.102:6816/34937 conn(0x7f6d60031160 legacy=0x7f6d60033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:50.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.544+0000 7f6d7c2c9640 1 -- 172.21.15.102:0/3247897518 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6d74128330 msgr2=0x7f6d7413aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:50.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.544+0000 7f6d7c2c9640 1 --2- 172.21.15.102:0/3247897518 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6d74128330 0x7f6d7413aa20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c031000 tx=0x7f6d6c032000 comp rx=0 tx=0).stop 2024-09-13T17:43:50.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.544+0000 7f6d7c2c9640 1 -- 172.21.15.102:0/3247897518 shutdown_connections 2024-09-13T17:43:50.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.544+0000 7f6d7c2c9640 1 --2- 172.21.15.102:0/3247897518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d7412bd80 0x7f6d7413b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:50.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.544+0000 7f6d7c2c9640 1 --2- 172.21.15.102:0/3247897518 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6d74128c40 0x7f6d7413af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:50.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.544+0000 7f6d7c2c9640 1 --2- 172.21.15.102:0/3247897518 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6d74128330 0x7f6d7413aa20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:50.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.544+0000 7f6d7c2c9640 1 -- 172.21.15.102:0/3247897518 >> 172.21.15.102:0/3247897518 conn(0x7f6d7409e820 msgr2=0x7f6d7412afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:50.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.544+0000 7f6d7c2c9640 1 -- 172.21.15.102:0/3247897518 shutdown_connections 2024-09-13T17:43:50.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.544+0000 7f6d7c2c9640 1 -- 172.21.15.102:0/3247897518 wait complete. 2024-09-13T17:43:50.558 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-13T17:43:50.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.654+0000 7fea07f1d640 1 Processor -- start 2024-09-13T17:43:50.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.654+0000 7fea07f1d640 1 -- start start 2024-09-13T17:43:50.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.655+0000 7fea07f1d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fea0009a620 0x7fea00096c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:50.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.655+0000 7fea07f1d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fea00097570 0x7fea0012d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:50.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.655+0000 7fea07f1d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea00098040 con 0x7fea00097170 2024-09-13T17:43:50.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.655+0000 7fea07f1d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea0009a9f0 con 0x7fea0009a620 2024-09-13T17:43:50.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.655+0000 7fea07f1d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea000982a0 con 0x7fea00097570 2024-09-13T17:43:50.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.655+0000 7fea05cb9640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fea0009a620 0x7fea00096c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:50.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.655+0000 7fea064ba640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fea00097570 0x7fea0012d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:50.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.655+0000 7fea054b8640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fea00097170 0x7fea0012d180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36476/0 (socket says 172.21.15.102:36476) 2024-09-13T17:43:50.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.655+0000 7fea054b8640 1 -- 172.21.15.102:0/989460709 learned_addr learned my addr 172.21.15.102:0/989460709 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:50.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.655+0000 7fea05cb9640 1 --2- 172.21.15.102:0/989460709 >> v2:172.21.15.102:3300/0 conn(0x7fea0009a620 0x7fea00096c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57878/0 (socket says 172.21.15.102:57878) 2024-09-13T17:43:50.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.655+0000 7fea05cb9640 1 -- 172.21.15.102:0/989460709 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fea00097570 msgr2=0x7fea0012d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:50.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.655+0000 7fea05cb9640 1 --2- 172.21.15.102:0/989460709 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fea00097570 0x7fea0012d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:50.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.655+0000 7fea05cb9640 1 -- 172.21.15.102:0/989460709 >> v1:172.21.15.102:6789/0 conn(0x7fea00097170 legacy=0x7fea0012d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:50.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.655+0000 7fea05cb9640 1 -- 172.21.15.102:0/989460709 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea0009ad50 con 0x7fea0009a620 2024-09-13T17:43:50.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.655+0000 7fea04cb7640 1 -- 172.21.15.102:0/989460709 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3747065620 0 0) 0x7fea00098040 con 0x7fea00097170 2024-09-13T17:43:50.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.655+0000 7fea064ba640 1 --2- 172.21.15.102:0/989460709 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fea00097570 0x7fea0012d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:50.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.656+0000 7fea05cb9640 1 --2- 172.21.15.102:0/989460709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea0009a620 0x7fea00096c30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc009e60 tx=0x7fe9fc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b130f489454bcf31 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:50.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.656+0000 7fea04cb7640 1 -- 172.21.15.102:0/989460709 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc007550 con 0x7fea0009a620 2024-09-13T17:43:50.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.656+0000 7fea04cb7640 1 -- 172.21.15.102:0/989460709 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9fc02d050 con 0x7fea0009a620 2024-09-13T17:43:50.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.656+0000 7fea04cb7640 1 -- 172.21.15.102:0/989460709 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc0078e0 con 0x7fea0009a620 2024-09-13T17:43:50.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.656+0000 7fea07f1d640 1 -- 172.21.15.102:0/989460709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea0009a620 msgr2=0x7fea00096c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:50.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.656+0000 7fea07f1d640 1 --2- 172.21.15.102:0/989460709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea0009a620 0x7fea00096c30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc009e60 tx=0x7fe9fc009780 comp rx=0 tx=0).stop 2024-09-13T17:43:50.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.657+0000 7fea07f1d640 1 -- 172.21.15.102:0/989460709 shutdown_connections 2024-09-13T17:43:50.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.657+0000 7fea07f1d640 1 --2- 172.21.15.102:0/989460709 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fea00097570 0x7fea0012d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:50.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.657+0000 7fea07f1d640 1 --2- 172.21.15.102:0/989460709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea0009a620 0x7fea00096c30 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:50.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.657+0000 7fea07f1d640 1 -- 172.21.15.102:0/989460709 >> 172.21.15.102:0/989460709 conn(0x7fea0009e6b0 msgr2=0x7fea0009eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:50.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.657+0000 7fea07f1d640 1 -- 172.21.15.102:0/989460709 shutdown_connections 2024-09-13T17:43:50.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.657+0000 7fea07f1d640 1 -- 172.21.15.102:0/989460709 wait complete. 2024-09-13T17:43:50.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.657+0000 7fea07f1d640 1 Processor -- start 2024-09-13T17:43:50.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.657+0000 7fea07f1d640 1 -- start start 2024-09-13T17:43:50.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.658+0000 7fea07f1d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea00097170 0x7fea00121aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:50.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.658+0000 7fea07f1d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fea00097570 0x7fea00098440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:50.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.658+0000 7fea07f1d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fea0009a620 0x7fea00098980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:50.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.658+0000 7fea07f1d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea00120e20 con 0x7fea00097170 2024-09-13T17:43:50.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.658+0000 7fea07f1d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea00121000 con 0x7fea0009a620 2024-09-13T17:43:50.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.658+0000 7fea07f1d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea001215a0 con 0x7fea00097570 2024-09-13T17:43:50.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.658+0000 7fea05cb9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea00097170 0x7fea00121aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:50.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.658+0000 7fea05cb9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea00097170 0x7fea00121aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57894/0 (socket says 172.21.15.102:57894) 2024-09-13T17:43:50.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.658+0000 7fea05cb9640 1 -- 172.21.15.102:0/2244645272 learned_addr learned my addr 172.21.15.102:0/2244645272 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:50.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.658+0000 7fea064ba640 1 --2- 172.21.15.102:0/2244645272 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fea0009a620 0x7fea00098980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:50.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.658+0000 7fea054b8640 1 --2- 172.21.15.102:0/2244645272 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fea00097570 0x7fea00098440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:50.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.658+0000 7fea064ba640 1 -- 172.21.15.102:0/2244645272 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fea00097570 msgr2=0x7fea00098440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:50.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.658+0000 7fea064ba640 1 --2- 172.21.15.102:0/2244645272 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fea00097570 0x7fea00098440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:50.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.658+0000 7fea064ba640 1 -- 172.21.15.102:0/2244645272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea00097170 msgr2=0x7fea00121aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:50.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.658+0000 7fea064ba640 1 --2- 172.21.15.102:0/2244645272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea00097170 0x7fea00121aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:50.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.658+0000 7fea064ba640 1 -- 172.21.15.102:0/2244645272 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f8009390 con 0x7fea0009a620 2024-09-13T17:43:50.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.658+0000 7fea05cb9640 1 --2- 172.21.15.102:0/2244645272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea00097170 0x7fea00121aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:50.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.658+0000 7fea064ba640 1 --2- 172.21.15.102:0/2244645272 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fea0009a620 0x7fea00098980 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800e980 tx=0x7fe9f80090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:50.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.659+0000 7fe9f5ffb640 1 -- 172.21.15.102:0/2244645272 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f80095e0 con 0x7fea0009a620 2024-09-13T17:43:50.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.659+0000 7fe9f5ffb640 1 -- 172.21.15.102:0/2244645272 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f800c940 con 0x7fea0009a620 2024-09-13T17:43:50.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.659+0000 7fea07f1d640 1 -- 172.21.15.102:0/2244645272 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9fc009a20 con 0x7fea0009a620 2024-09-13T17:43:50.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.659+0000 7fe9f5ffb640 1 -- 172.21.15.102:0/2244645272 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f800cd00 con 0x7fea0009a620 2024-09-13T17:43:50.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.659+0000 7fea07f1d640 1 -- 172.21.15.102:0/2244645272 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea00099f80 con 0x7fea0009a620 2024-09-13T17:43:50.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.660+0000 7fe9f5ffb640 1 -- 172.21.15.102:0/2244645272 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fe9f8013680 con 0x7fea0009a620 2024-09-13T17:43:50.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.660+0000 7fe9f5ffb640 1 -- 172.21.15.102:0/2244645272 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7fe9f8042160 con 0x7fea0009a620 2024-09-13T17:43:50.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.660+0000 7fea07f1d640 1 -- 172.21.15.102:0/2244645272 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea00064000 con 0x7fea0009a620 2024-09-13T17:43:50.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.664+0000 7fe9f5ffb640 1 -- 172.21.15.102:0/2244645272 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fe9f801ab40 con 0x7fea0009a620 2024-09-13T17:43:50.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.816+0000 7fea07f1d640 1 -- 172.21.15.102:0/2244645272 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fea0006e5e0 con 0x7fea0009a620 2024-09-13T17:43:50.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.821+0000 7fe9f5ffb640 1 -- 172.21.15.102:0/2244645272 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v6)=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe9f8020030 con 0x7fea0009a620 2024-09-13T17:43:50.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.823+0000 7fea07f1d640 1 -- 172.21.15.102:0/2244645272 >> v1:172.21.15.102:6816/34937 conn(0x7fe9e4031160 legacy=0x7fe9e4033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:50.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.823+0000 7fea07f1d640 1 -- 172.21.15.102:0/2244645272 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fea0009a620 msgr2=0x7fea00098980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:50.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.823+0000 7fea07f1d640 1 --2- 172.21.15.102:0/2244645272 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fea0009a620 0x7fea00098980 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800e980 tx=0x7fe9f80090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:50.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.824+0000 7fea07f1d640 1 -- 172.21.15.102:0/2244645272 shutdown_connections 2024-09-13T17:43:50.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.824+0000 7fea07f1d640 1 --2- 172.21.15.102:0/2244645272 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fea0009a620 0x7fea00098980 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:50.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.824+0000 7fea07f1d640 1 --2- 172.21.15.102:0/2244645272 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fea00097570 0x7fea00098440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:50.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.824+0000 7fea07f1d640 1 --2- 172.21.15.102:0/2244645272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea00097170 0x7fea00121aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:50.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.824+0000 7fea07f1d640 1 -- 172.21.15.102:0/2244645272 >> 172.21.15.102:0/2244645272 conn(0x7fea0009e6b0 msgr2=0x7fea0006db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:50.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.824+0000 7fea07f1d640 1 -- 172.21.15.102:0/2244645272 shutdown_connections 2024-09-13T17:43:50.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:50.824+0000 7fea07f1d640 1 -- 172.21.15.102:0/2244645272 wait complete. 2024-09-13T17:43:50.836 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T17:43:50.836 DEBUG:teuthology.orchestra.run.smithi102:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T17:43:51.101 DEBUG:teuthology.orchestra.run.smithi171:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T17:43:51.146 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T17:43:51.146 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T17:43:51.146 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:43:51.205 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T17:43:51.205 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:43:51.264 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:43:51.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.361+0000 7f307919d640 1 Processor -- start 2024-09-13T17:43:51.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.361+0000 7f307919d640 1 -- start start 2024-09-13T17:43:51.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.361+0000 7f307919d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3074128e10 0x7f3074130b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:51.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.361+0000 7f307919d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f30741310b0 0x7f3074133490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:51.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.361+0000 7f307919d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3074097be0 con 0x7f3074128330 2024-09-13T17:43:51.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.361+0000 7f307919d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3074129200 con 0x7f3074128e10 2024-09-13T17:43:51.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.361+0000 7f307919d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3074098e70 con 0x7f30741310b0 2024-09-13T17:43:51.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.361+0000 7f306bfff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f3074128e10 0x7f3074130b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:51.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.361+0000 7f307359e640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f30741310b0 0x7f3074133490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:51.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.361+0000 7f3072d9d640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f3074128330 0x7f3074128700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36492/0 (socket says 172.21.15.102:36492) 2024-09-13T17:43:51.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.361+0000 7f3072d9d640 1 -- 172.21.15.102:0/1738121084 learned_addr learned my addr 172.21.15.102:0/1738121084 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:51.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.362+0000 7f306bfff640 1 -- 172.21.15.102:0/1738121084 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f30741310b0 msgr2=0x7f3074133490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:51.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.362+0000 7f306bfff640 1 --2- 172.21.15.102:0/1738121084 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f30741310b0 0x7f3074133490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:51.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.362+0000 7f306bfff640 1 -- 172.21.15.102:0/1738121084 >> v1:172.21.15.102:6789/0 conn(0x7f3074128330 legacy=0x7f3074128700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:51.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.362+0000 7f306bfff640 1 -- 172.21.15.102:0/1738121084 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3074098310 con 0x7f3074128e10 2024-09-13T17:43:51.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.362+0000 7f306bfff640 1 --2- 172.21.15.102:0/1738121084 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3074128e10 0x7f3074130b70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f305c009850 tx=0x7f305c0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348bc9ecb8bd5eb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:51.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.363+0000 7f307259c640 1 -- 172.21.15.102:0/1738121084 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c02ad90 con 0x7f3074128e10 2024-09-13T17:43:51.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.363+0000 7f307259c640 1 -- 172.21.15.102:0/1738121084 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305c032440 con 0x7f3074128e10 2024-09-13T17:43:51.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.363+0000 7f307259c640 1 -- 172.21.15.102:0/1738121084 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c0327e0 con 0x7f3074128e10 2024-09-13T17:43:51.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.363+0000 7f307919d640 1 -- 172.21.15.102:0/1738121084 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3074128e10 msgr2=0x7f3074130b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:51.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.363+0000 7f307919d640 1 --2- 172.21.15.102:0/1738121084 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3074128e10 0x7f3074130b70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f305c009850 tx=0x7f305c0090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:51.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.363+0000 7f307919d640 1 -- 172.21.15.102:0/1738121084 shutdown_connections 2024-09-13T17:43:51.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.363+0000 7f307919d640 1 --2- 172.21.15.102:0/1738121084 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f30741310b0 0x7f3074133490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:51.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.363+0000 7f307919d640 1 --2- 172.21.15.102:0/1738121084 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3074128e10 0x7f3074130b70 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:51.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.363+0000 7f307919d640 1 -- 172.21.15.102:0/1738121084 >> 172.21.15.102:0/1738121084 conn(0x7f307409e820 msgr2=0x7f307409ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:51.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.363+0000 7f307919d640 1 -- 172.21.15.102:0/1738121084 shutdown_connections 2024-09-13T17:43:51.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.363+0000 7f307919d640 1 -- 172.21.15.102:0/1738121084 wait complete. 2024-09-13T17:43:51.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.364+0000 7f307919d640 1 Processor -- start 2024-09-13T17:43:51.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.364+0000 7f307919d640 1 -- start start 2024-09-13T17:43:51.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.364+0000 7f307919d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3074128330 0x7f307412a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:51.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.364+0000 7f307919d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3074128e10 0x7f307406bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:51.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.364+0000 7f307919d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f30741310b0 0x7f307406c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:51.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.364+0000 7f307919d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307406dc00 con 0x7f3074128330 2024-09-13T17:43:51.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.364+0000 7f307919d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3074129ba0 con 0x7f30741310b0 2024-09-13T17:43:51.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.364+0000 7f307919d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3074130b90 con 0x7f3074128e10 2024-09-13T17:43:51.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.364+0000 7f306bfff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3074128e10 0x7f307406bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:51.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.364+0000 7f3072d9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3074128330 0x7f307412a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:51.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.364+0000 7f306bfff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3074128e10 0x7f307406bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:43520/0 (socket says 172.21.15.102:43520) 2024-09-13T17:43:51.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.364+0000 7f306bfff640 1 -- 172.21.15.102:0/2482852904 learned_addr learned my addr 172.21.15.102:0/2482852904 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:51.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.364+0000 7f307359e640 1 --2- 172.21.15.102:0/2482852904 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f30741310b0 0x7f307406c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:51.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.365+0000 7f306bfff640 1 -- 172.21.15.102:0/2482852904 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f30741310b0 msgr2=0x7f307406c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:51.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.365+0000 7f306bfff640 1 --2- 172.21.15.102:0/2482852904 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f30741310b0 0x7f307406c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:51.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.365+0000 7f306bfff640 1 -- 172.21.15.102:0/2482852904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3074128330 msgr2=0x7f307412a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:51.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.365+0000 7f306bfff640 1 --2- 172.21.15.102:0/2482852904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3074128330 0x7f307412a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:51.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.365+0000 7f306bfff640 1 -- 172.21.15.102:0/2482852904 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f305c009390 con 0x7f3074128e10 2024-09-13T17:43:51.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.365+0000 7f306bfff640 1 --2- 172.21.15.102:0/2482852904 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3074128e10 0x7f307406bfa0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f305c009360 tx=0x7f305c032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:51.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.365+0000 7f306affd640 1 -- 172.21.15.102:0/2482852904 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c031de0 con 0x7f3074128e10 2024-09-13T17:43:51.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.365+0000 7f306affd640 1 -- 172.21.15.102:0/2482852904 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305c03d420 con 0x7f3074128e10 2024-09-13T17:43:51.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.365+0000 7f306affd640 1 -- 172.21.15.102:0/2482852904 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c03d7e0 con 0x7f3074128e10 2024-09-13T17:43:51.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.365+0000 7f307919d640 1 -- 172.21.15.102:0/2482852904 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f307406e200 con 0x7f3074128e10 2024-09-13T17:43:51.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.365+0000 7f307919d640 1 -- 172.21.15.102:0/2482852904 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f307406ca20 con 0x7f3074128e10 2024-09-13T17:43:51.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.366+0000 7f306affd640 1 -- 172.21.15.102:0/2482852904 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f305c039040 con 0x7f3074128e10 2024-09-13T17:43:51.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.366+0000 7f306affd640 1 -- 172.21.15.102:0/2482852904 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f305c0624e0 con 0x7f3074128e10 2024-09-13T17:43:51.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.366+0000 7f307919d640 1 -- 172.21.15.102:0/2482852904 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3074064000 con 0x7f3074128e10 2024-09-13T17:43:51.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.370+0000 7f306affd640 1 -- 172.21.15.102:0/2482852904 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f305c037030 con 0x7f3074128e10 2024-09-13T17:43:51.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.525+0000 7f307919d640 1 -- 172.21.15.102:0/2482852904 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3074129f10 con 0x7f3074128e10 2024-09-13T17:43:51.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.525+0000 7f306affd640 1 -- 172.21.15.102:0/2482852904 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+11660 (secure 0 0 0) 0x7f305c037030 con 0x7f3074128e10 2024-09-13T17:43:51.527 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:43:51.527 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":23,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:42:48.385896+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:04.672265+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:05.583134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.17","mappings":[{"from":3,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]},{"pgid":"4.25","mappings":[{"from":3,"to":2}]},{"pgid":"4.33","mappings":[{"from":3,"to":0}]},{"pgid":"4.3a","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:43:51.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.527+0000 7f307919d640 1 -- 172.21.15.102:0/2482852904 >> v1:172.21.15.102:6816/34937 conn(0x7f3054031160 legacy=0x7f3054033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:51.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.527+0000 7f307919d640 1 -- 172.21.15.102:0/2482852904 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3074128e10 msgr2=0x7f307406bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:51.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.527+0000 7f307919d640 1 --2- 172.21.15.102:0/2482852904 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3074128e10 0x7f307406bfa0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f305c009360 tx=0x7f305c032000 comp rx=0 tx=0).stop 2024-09-13T17:43:51.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.527+0000 7f307919d640 1 -- 172.21.15.102:0/2482852904 shutdown_connections 2024-09-13T17:43:51.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.527+0000 7f307919d640 1 --2- 172.21.15.102:0/2482852904 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f30741310b0 0x7f307406c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:51.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.527+0000 7f307919d640 1 --2- 172.21.15.102:0/2482852904 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3074128e10 0x7f307406bfa0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:51.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.527+0000 7f307919d640 1 --2- 172.21.15.102:0/2482852904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3074128330 0x7f307412a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:51.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.527+0000 7f307919d640 1 -- 172.21.15.102:0/2482852904 >> 172.21.15.102:0/2482852904 conn(0x7f307409e820 msgr2=0x7f30741265c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:51.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.527+0000 7f307919d640 1 -- 172.21.15.102:0/2482852904 shutdown_connections 2024-09-13T17:43:51.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.527+0000 7f307919d640 1 -- 172.21.15.102:0/2482852904 wait complete. 2024-09-13T17:43:51.540 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:54.442612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T17:41:59.076782+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 16, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:42:04.672265+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:42:05.583134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:43:51.541 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:43:51.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.635+0000 7f632979d640 1 Processor -- start 2024-09-13T17:43:51.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.635+0000 7f632979d640 1 -- start start 2024-09-13T17:43:51.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.635+0000 7f632979d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f63240982e0 0x7f632409e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:51.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.636+0000 7f632979d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f632409e5d0 0x7f63240989a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:51.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.636+0000 7f632979d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63240986b0 con 0x7f6324098ee0 2024-09-13T17:43:51.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.636+0000 7f632979d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6324099a50 con 0x7f632409e5d0 2024-09-13T17:43:51.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.636+0000 7f632979d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6324097820 con 0x7f63240982e0 2024-09-13T17:43:51.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.636+0000 7f63227fc640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f632409e5d0 0x7f63240989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:51.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.636+0000 7f6322ffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f63240982e0 0x7f632409e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:51.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.636+0000 7f63237fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f6324098ee0 0x7f6324099340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:36494/0 (socket says 172.21.15.102:36494) 2024-09-13T17:43:51.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.636+0000 7f63237fe640 1 -- 172.21.15.102:0/4236710713 learned_addr learned my addr 172.21.15.102:0/4236710713 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:51.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.636+0000 7f6322ffd640 1 -- 172.21.15.102:0/4236710713 >> v2:172.21.15.102:3300/0 conn(0x7f632409e5d0 msgr2=0x7f63240989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:51.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.636+0000 7f6322ffd640 1 --2- 172.21.15.102:0/4236710713 >> v2:172.21.15.102:3300/0 conn(0x7f632409e5d0 0x7f63240989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:51.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.636+0000 7f6322ffd640 1 -- 172.21.15.102:0/4236710713 >> v1:172.21.15.102:6789/0 conn(0x7f6324098ee0 legacy=0x7f6324099340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:51.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.636+0000 7f6322ffd640 1 -- 172.21.15.102:0/4236710713 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6324099de0 con 0x7f63240982e0 2024-09-13T17:43:51.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.636+0000 7f63227fc640 1 --2- 172.21.15.102:0/4236710713 >> v2:172.21.15.102:3300/0 conn(0x7f632409e5d0 0x7f63240989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:51.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.637+0000 7f6322ffd640 1 --2- 172.21.15.102:0/4236710713 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f63240982e0 0x7f632409e090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6314009850 tx=0x7f63140090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49497bb4c98c7172 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:51.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.637+0000 7f6321ffb640 1 -- 172.21.15.102:0/4236710713 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631402ad00 con 0x7f63240982e0 2024-09-13T17:43:51.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.637+0000 7f6321ffb640 1 -- 172.21.15.102:0/4236710713 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f631402ae60 con 0x7f63240982e0 2024-09-13T17:43:51.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.637+0000 7f6321ffb640 1 -- 172.21.15.102:0/4236710713 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6314032670 con 0x7f63240982e0 2024-09-13T17:43:51.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.637+0000 7f632979d640 1 -- 172.21.15.102:0/4236710713 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f63240982e0 msgr2=0x7f632409e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:51.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.637+0000 7f632979d640 1 --2- 172.21.15.102:0/4236710713 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f63240982e0 0x7f632409e090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6314009850 tx=0x7f63140090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:51.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.637+0000 7f632979d640 1 -- 172.21.15.102:0/4236710713 shutdown_connections 2024-09-13T17:43:51.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.637+0000 7f632979d640 1 --2- 172.21.15.102:0/4236710713 >> v2:172.21.15.102:3300/0 conn(0x7f632409e5d0 0x7f63240989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:51.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.637+0000 7f632979d640 1 --2- 172.21.15.102:0/4236710713 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f63240982e0 0x7f632409e090 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:51.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.637+0000 7f632979d640 1 -- 172.21.15.102:0/4236710713 >> 172.21.15.102:0/4236710713 conn(0x7f632405e620 msgr2=0x7f632409f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:51.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.637+0000 7f632979d640 1 -- 172.21.15.102:0/4236710713 shutdown_connections 2024-09-13T17:43:51.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.637+0000 7f632979d640 1 -- 172.21.15.102:0/4236710713 wait complete. 2024-09-13T17:43:51.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.638+0000 7f632979d640 1 Processor -- start 2024-09-13T17:43:51.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.638+0000 7f632979d640 1 -- start start 2024-09-13T17:43:51.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.639+0000 7f632979d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f63240982e0 0x7f6324118ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:51.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.639+0000 7f632979d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6324098ee0 0x7f6324119220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:51.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.639+0000 7f632979d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f632409e5d0 0x7f632413c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:51.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.639+0000 7f632979d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632406b650 con 0x7f632409e5d0 2024-09-13T17:43:51.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.639+0000 7f632979d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6324096dc0 con 0x7f6324098ee0 2024-09-13T17:43:51.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.639+0000 7f632979d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6324118a00 con 0x7f63240982e0 2024-09-13T17:43:51.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.639+0000 7f6322ffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f63240982e0 0x7f6324118ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:51.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.639+0000 7f6322ffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f63240982e0 0x7f6324118ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:43542/0 (socket says 172.21.15.102:43542) 2024-09-13T17:43:51.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.639+0000 7f6322ffd640 1 -- 172.21.15.102:0/2215892622 learned_addr learned my addr 172.21.15.102:0/2215892622 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:51.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.639+0000 7f63227fc640 1 --2- 172.21.15.102:0/2215892622 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6324098ee0 0x7f6324119220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:51.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.639+0000 7f63237fe640 1 --2- 172.21.15.102:0/2215892622 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f632409e5d0 0x7f632413c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:51.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.639+0000 7f63227fc640 1 -- 172.21.15.102:0/2215892622 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f63240982e0 msgr2=0x7f6324118ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:51.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.639+0000 7f63227fc640 1 --2- 172.21.15.102:0/2215892622 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f63240982e0 0x7f6324118ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:51.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.639+0000 7f63227fc640 1 -- 172.21.15.102:0/2215892622 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f632409e5d0 msgr2=0x7f632413c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:51.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.639+0000 7f63227fc640 1 --2- 172.21.15.102:0/2215892622 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f632409e5d0 0x7f632413c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:51.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.639+0000 7f63227fc640 1 -- 172.21.15.102:0/2215892622 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6318009390 con 0x7f6324098ee0 2024-09-13T17:43:51.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.639+0000 7f63227fc640 1 --2- 172.21.15.102:0/2215892622 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6324098ee0 0x7f6324119220 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f631800ea50 tx=0x7f63180090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:51.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.640+0000 7f630affd640 1 -- 172.21.15.102:0/2215892622 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631800ca10 con 0x7f6324098ee0 2024-09-13T17:43:51.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.640+0000 7f630affd640 1 -- 172.21.15.102:0/2215892622 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f631800cb70 con 0x7f6324098ee0 2024-09-13T17:43:51.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.640+0000 7f632979d640 1 -- 172.21.15.102:0/2215892622 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6314009390 con 0x7f6324098ee0 2024-09-13T17:43:51.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.640+0000 7f630affd640 1 -- 172.21.15.102:0/2215892622 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6318013470 con 0x7f6324098ee0 2024-09-13T17:43:51.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.640+0000 7f632979d640 1 -- 172.21.15.102:0/2215892622 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f632411c790 con 0x7f6324098ee0 2024-09-13T17:43:51.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.640+0000 7f630affd640 1 -- 172.21.15.102:0/2215892622 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f6318013860 con 0x7f6324098ee0 2024-09-13T17:43:51.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.641+0000 7f630affd640 1 -- 172.21.15.102:0/2215892622 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f63180424e0 con 0x7f6324098ee0 2024-09-13T17:43:51.645 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.641+0000 7f632979d640 1 -- 172.21.15.102:0/2215892622 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6324064600 con 0x7f6324098ee0 2024-09-13T17:43:51.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.645+0000 7f630affd640 1 -- 172.21.15.102:0/2215892622 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f6318023ad0 con 0x7f6324098ee0 2024-09-13T17:43:51.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.794+0000 7f632979d640 1 -- 172.21.15.102:0/2215892622 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f63240687a0 con 0x7f6324098ee0 2024-09-13T17:43:51.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.794+0000 7f630affd640 1 -- 172.21.15.102:0/2215892622 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f63180232f0 con 0x7f6324098ee0 2024-09-13T17:43:51.796 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-09-13T17:43:51.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.796+0000 7f632979d640 1 -- 172.21.15.102:0/2215892622 >> v1:172.21.15.102:6816/34937 conn(0x7f6310031160 legacy=0x7f6310033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:51.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.796+0000 7f632979d640 1 -- 172.21.15.102:0/2215892622 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6324098ee0 msgr2=0x7f6324119220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:51.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.796+0000 7f632979d640 1 --2- 172.21.15.102:0/2215892622 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6324098ee0 0x7f6324119220 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f631800ea50 tx=0x7f63180090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:51.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.796+0000 7f632979d640 1 -- 172.21.15.102:0/2215892622 shutdown_connections 2024-09-13T17:43:51.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.796+0000 7f632979d640 1 --2- 172.21.15.102:0/2215892622 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f632409e5d0 0x7f632413c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:51.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.796+0000 7f632979d640 1 --2- 172.21.15.102:0/2215892622 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6324098ee0 0x7f6324119220 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:51.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.796+0000 7f632979d640 1 --2- 172.21.15.102:0/2215892622 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f63240982e0 0x7f6324118ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:51.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.796+0000 7f632979d640 1 -- 172.21.15.102:0/2215892622 >> 172.21.15.102:0/2215892622 conn(0x7f632405e620 msgr2=0x7f6324067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:51.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.796+0000 7f632979d640 1 -- 172.21.15.102:0/2215892622 shutdown_connections 2024-09-13T17:43:51.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.796+0000 7f632979d640 1 -- 172.21.15.102:0/2215892622 wait complete. 2024-09-13T17:43:51.808 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-13T17:43:51.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.903+0000 7f0fd924f640 1 Processor -- start 2024-09-13T17:43:51.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.903+0000 7f0fd924f640 1 -- start start 2024-09-13T17:43:51.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.903+0000 7f0fd924f640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0fd4097530 0x7f0fd4097900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:51.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.903+0000 7f0fd924f640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0fd4098210 0x7f0fd4133640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:51.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.903+0000 7f0fd924f640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fd4098c50 con 0x7f0fd4097e40 2024-09-13T17:43:51.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.903+0000 7f0fd924f640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd406ed00 con 0x7f0fd4098210 2024-09-13T17:43:51.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.903+0000 7f0fd924f640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd4068bd0 con 0x7f0fd4097530 2024-09-13T17:43:51.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.903+0000 7f0fd2d9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0fd4097530 0x7f0fd4097900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:51.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.903+0000 7f0fd359e640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0fd4098210 0x7f0fd4133640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:51.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.903+0000 7f0fd2d9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0fd4097530 0x7f0fd4097900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:51084/0 (socket says 172.21.15.102:51084) 2024-09-13T17:43:51.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.903+0000 7f0fd359e640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0fd4098210 0x7f0fd4133640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57954/0 (socket says 172.21.15.102:57954) 2024-09-13T17:43:51.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.903+0000 7f0fd2d9d640 1 -- 172.21.15.102:0/1129125793 learned_addr learned my addr 172.21.15.102:0/1129125793 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:51.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.904+0000 7f0fd359e640 1 -- 172.21.15.102:0/1129125793 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0fd4097530 msgr2=0x7f0fd4097900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:51.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.904+0000 7f0fd359e640 1 --2- 172.21.15.102:0/1129125793 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0fd4097530 0x7f0fd4097900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:51.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.904+0000 7f0fd359e640 1 -- 172.21.15.102:0/1129125793 >> v1:172.21.15.102:6789/0 conn(0x7f0fd4097e40 legacy=0x7f0fd412ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:51.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.904+0000 7f0fd259c640 1 -- 172.21.15.102:0/1129125793 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2026257802 0 0) 0x7f0fd4098c50 con 0x7f0fd4097e40 2024-09-13T17:43:51.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.904+0000 7f0fd359e640 1 -- 172.21.15.102:0/1129125793 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd4099040 con 0x7f0fd4098210 2024-09-13T17:43:51.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.904+0000 7f0fd2d9d640 1 --2- 172.21.15.102:0/1129125793 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0fd4097530 0x7f0fd4097900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:51.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.904+0000 7f0fd359e640 1 --2- 172.21.15.102:0/1129125793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fd4098210 0x7f0fd4133640 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc009e60 tx=0x7f0fcc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840775097b0e0438 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:51.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.904+0000 7f0fd259c640 1 -- 172.21.15.102:0/1129125793 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc007550 con 0x7f0fd4098210 2024-09-13T17:43:51.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.905+0000 7f0fd259c640 1 -- 172.21.15.102:0/1129125793 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fcc02d050 con 0x7f0fd4098210 2024-09-13T17:43:51.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.905+0000 7f0fd259c640 1 -- 172.21.15.102:0/1129125793 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc0078e0 con 0x7f0fd4098210 2024-09-13T17:43:51.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.905+0000 7f0fd924f640 1 -- 172.21.15.102:0/1129125793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fd4098210 msgr2=0x7f0fd4133640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:51.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.905+0000 7f0fd924f640 1 --2- 172.21.15.102:0/1129125793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fd4098210 0x7f0fd4133640 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc009e60 tx=0x7f0fcc009780 comp rx=0 tx=0).stop 2024-09-13T17:43:51.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.905+0000 7f0fd924f640 1 -- 172.21.15.102:0/1129125793 shutdown_connections 2024-09-13T17:43:51.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.905+0000 7f0fd924f640 1 --2- 172.21.15.102:0/1129125793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fd4098210 0x7f0fd4133640 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:51.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.905+0000 7f0fd924f640 1 --2- 172.21.15.102:0/1129125793 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0fd4097530 0x7f0fd4097900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:51.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.905+0000 7f0fd924f640 1 -- 172.21.15.102:0/1129125793 >> 172.21.15.102:0/1129125793 conn(0x7f0fd409e720 msgr2=0x7f0fd409eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:51.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.905+0000 7f0fd924f640 1 -- 172.21.15.102:0/1129125793 shutdown_connections 2024-09-13T17:43:51.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.905+0000 7f0fd924f640 1 -- 172.21.15.102:0/1129125793 wait complete. 2024-09-13T17:43:51.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.906+0000 7f0fd924f640 1 Processor -- start 2024-09-13T17:43:51.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.906+0000 7f0fd924f640 1 -- start start 2024-09-13T17:43:51.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.906+0000 7f0fd924f640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0fd4097530 0x7f0fd412e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:51.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.906+0000 7f0fd924f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fd4097e40 0x7f0fd411a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:51.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.906+0000 7f0fd924f640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0fd4098210 0x7f0fd411ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:51.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.906+0000 7f0fd924f640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd411c6f0 con 0x7f0fd4097e40 2024-09-13T17:43:51.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.906+0000 7f0fd924f640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd411d130 con 0x7f0fd4097530 2024-09-13T17:43:51.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.906+0000 7f0fd924f640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd411c1b0 con 0x7f0fd4098210 2024-09-13T17:43:51.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.906+0000 7f0fd2d9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0fd4097530 0x7f0fd412e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:51.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.906+0000 7f0fca59c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fd4097e40 0x7f0fd411a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:51.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.907+0000 7f0fd2d9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0fd4097530 0x7f0fd412e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:51096/0 (socket says 172.21.15.102:51096) 2024-09-13T17:43:51.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.907+0000 7f0fd2d9d640 1 -- 172.21.15.102:0/182225723 learned_addr learned my addr 172.21.15.102:0/182225723 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:51.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.907+0000 7f0fd359e640 1 --2- 172.21.15.102:0/182225723 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0fd4098210 0x7f0fd411ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:51.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.907+0000 7f0fd2d9d640 1 -- 172.21.15.102:0/182225723 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0fd4098210 msgr2=0x7f0fd411ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:51.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.907+0000 7f0fd2d9d640 1 --2- 172.21.15.102:0/182225723 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0fd4098210 0x7f0fd411ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:51.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.907+0000 7f0fd2d9d640 1 -- 172.21.15.102:0/182225723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fd4097e40 msgr2=0x7f0fd411a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:51.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.907+0000 7f0fd2d9d640 1 --2- 172.21.15.102:0/182225723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fd4097e40 0x7f0fd411a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:51.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.907+0000 7f0fd2d9d640 1 -- 172.21.15.102:0/182225723 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc400b6f0 con 0x7f0fd4097530 2024-09-13T17:43:51.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.907+0000 7f0fca59c640 1 --2- 172.21.15.102:0/182225723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fd4097e40 0x7f0fd411a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:51.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.907+0000 7f0fd359e640 1 --2- 172.21.15.102:0/182225723 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0fd4098210 0x7f0fd411ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:51.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.907+0000 7f0fd2d9d640 1 --2- 172.21.15.102:0/182225723 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0fd4097530 0x7f0fd412e620 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40100f0 tx=0x7f0fc400f040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:51.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.907+0000 7f0fcb7fe640 1 -- 172.21.15.102:0/182225723 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4012040 con 0x7f0fd4097530 2024-09-13T17:43:51.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.907+0000 7f0fcb7fe640 1 -- 172.21.15.102:0/182225723 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc4010ed0 con 0x7f0fd4097530 2024-09-13T17:43:51.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.907+0000 7f0fd924f640 1 -- 172.21.15.102:0/182225723 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fcc009a20 con 0x7f0fd4097530 2024-09-13T17:43:51.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.907+0000 7f0fcb7fe640 1 -- 172.21.15.102:0/182225723 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4018620 con 0x7f0fd4097530 2024-09-13T17:43:51.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.907+0000 7f0fd924f640 1 -- 172.21.15.102:0/182225723 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd411e810 con 0x7f0fd4097530 2024-09-13T17:43:51.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.908+0000 7f0fcb7fe640 1 -- 172.21.15.102:0/182225723 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f0fc4018a50 con 0x7f0fd4097530 2024-09-13T17:43:51.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.908+0000 7f0fcb7fe640 1 -- 172.21.15.102:0/182225723 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f0fc4044630 con 0x7f0fd4097530 2024-09-13T17:43:51.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.908+0000 7f0fd924f640 1 -- 172.21.15.102:0/182225723 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd411def0 con 0x7f0fd4097530 2024-09-13T17:43:51.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:51.912+0000 7f0fcb7fe640 1 -- 172.21.15.102:0/182225723 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f0fc4017370 con 0x7f0fd4097530 2024-09-13T17:43:52.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.062+0000 7f0fd924f640 1 -- 172.21.15.102:0/182225723 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0fd4064000 con 0x7f0fd4097530 2024-09-13T17:43:52.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.063+0000 7f0fcb7fe640 1 -- 172.21.15.102:0/182225723 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0fc401faa0 con 0x7f0fd4097530 2024-09-13T17:43:52.065 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-09-13T17:43:52.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.065+0000 7f0fd924f640 1 -- 172.21.15.102:0/182225723 >> v1:172.21.15.102:6816/34937 conn(0x7f0fb4031160 legacy=0x7f0fb4033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:52.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.065+0000 7f0fd924f640 1 -- 172.21.15.102:0/182225723 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0fd4097530 msgr2=0x7f0fd412e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:52.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.065+0000 7f0fd924f640 1 --2- 172.21.15.102:0/182225723 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0fd4097530 0x7f0fd412e620 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40100f0 tx=0x7f0fc400f040 comp rx=0 tx=0).stop 2024-09-13T17:43:52.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.065+0000 7f0fd924f640 1 -- 172.21.15.102:0/182225723 shutdown_connections 2024-09-13T17:43:52.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.065+0000 7f0fd924f640 1 --2- 172.21.15.102:0/182225723 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0fd4098210 0x7f0fd411ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.065+0000 7f0fd924f640 1 --2- 172.21.15.102:0/182225723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fd4097e40 0x7f0fd411a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.065+0000 7f0fd924f640 1 --2- 172.21.15.102:0/182225723 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0fd4097530 0x7f0fd412e620 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.065+0000 7f0fd924f640 1 -- 172.21.15.102:0/182225723 >> 172.21.15.102:0/182225723 conn(0x7f0fd409e720 msgr2=0x7f0fd409c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:52.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.065+0000 7f0fd924f640 1 -- 172.21.15.102:0/182225723 shutdown_connections 2024-09-13T17:43:52.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.065+0000 7f0fd924f640 1 -- 172.21.15.102:0/182225723 wait complete. 2024-09-13T17:43:52.077 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:43:52.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.171+0000 7fa7c659d640 1 Processor -- start 2024-09-13T17:43:52.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.171+0000 7fa7c659d640 1 -- start start 2024-09-13T17:43:52.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.172+0000 7fa7c659d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa7c0096a70 0x7fa7c012c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:52.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.172+0000 7fa7c659d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa7c012ccd0 0x7fa7c0098b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:52.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.172+0000 7fa7c659d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7c0099540 con 0x7fa7c012c900 2024-09-13T17:43:52.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.172+0000 7fa7c659d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c0096e40 con 0x7fa7c012ccd0 2024-09-13T17:43:52.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.172+0000 7fa7c659d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c009a8e0 con 0x7fa7c0096a70 2024-09-13T17:43:52.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.172+0000 7fa7bffff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa7c0096a70 0x7fa7c012c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:52.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.172+0000 7fa7c4b3a640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa7c012ccd0 0x7fa7c0098b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:52.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.172+0000 7fa7b7fff640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fa7c012c900 0x7fa7c0098410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:58862/0 (socket says 172.21.15.102:58862) 2024-09-13T17:43:52.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.172+0000 7fa7b7fff640 1 -- 172.21.15.102:0/1114627380 learned_addr learned my addr 172.21.15.102:0/1114627380 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:52.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.172+0000 7fa7bffff640 1 -- 172.21.15.102:0/1114627380 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa7c012ccd0 msgr2=0x7fa7c0098b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:52.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.172+0000 7fa7bffff640 1 --2- 172.21.15.102:0/1114627380 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa7c012ccd0 0x7fa7c0098b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.172+0000 7fa7bffff640 1 -- 172.21.15.102:0/1114627380 >> v1:172.21.15.102:6789/0 conn(0x7fa7c012c900 legacy=0x7fa7c0098410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:52.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.172+0000 7fa7bffff640 1 -- 172.21.15.102:0/1114627380 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c0099760 con 0x7fa7c0096a70 2024-09-13T17:43:52.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.172+0000 7fa7bf7fe640 1 -- 172.21.15.102:0/1114627380 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1201928387 0 0) 0x7fa7c0099540 con 0x7fa7c012c900 2024-09-13T17:43:52.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.172+0000 7fa7c4b3a640 1 --2- 172.21.15.102:0/1114627380 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa7c012ccd0 0x7fa7c0098b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:52.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.173+0000 7fa7bffff640 1 --2- 172.21.15.102:0/1114627380 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa7c0096a70 0x7fa7c012c3c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa7b00098b0 tx=0x7fa7b00090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=133d424ae8a1e011 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:52.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.173+0000 7fa7bf7fe640 1 -- 172.21.15.102:0/1114627380 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b002acc0 con 0x7fa7c0096a70 2024-09-13T17:43:52.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.173+0000 7fa7bf7fe640 1 -- 172.21.15.102:0/1114627380 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b002ae20 con 0x7fa7c0096a70 2024-09-13T17:43:52.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.173+0000 7fa7bf7fe640 1 -- 172.21.15.102:0/1114627380 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0032670 con 0x7fa7c0096a70 2024-09-13T17:43:52.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.174+0000 7fa7c659d640 1 -- 172.21.15.102:0/1114627380 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa7c0096a70 msgr2=0x7fa7c012c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:52.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.174+0000 7fa7c659d640 1 --2- 172.21.15.102:0/1114627380 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa7c0096a70 0x7fa7c012c3c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa7b00098b0 tx=0x7fa7b00090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.174+0000 7fa7c659d640 1 -- 172.21.15.102:0/1114627380 shutdown_connections 2024-09-13T17:43:52.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.174+0000 7fa7c659d640 1 --2- 172.21.15.102:0/1114627380 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa7c012ccd0 0x7fa7c0098b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.174+0000 7fa7c659d640 1 --2- 172.21.15.102:0/1114627380 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa7c0096a70 0x7fa7c012c3c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.174+0000 7fa7c659d640 1 -- 172.21.15.102:0/1114627380 >> 172.21.15.102:0/1114627380 conn(0x7fa7c009e6b0 msgr2=0x7fa7c009eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:52.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.174+0000 7fa7c659d640 1 -- 172.21.15.102:0/1114627380 shutdown_connections 2024-09-13T17:43:52.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.174+0000 7fa7c659d640 1 -- 172.21.15.102:0/1114627380 wait complete. 2024-09-13T17:43:52.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.175+0000 7fa7c659d640 1 Processor -- start 2024-09-13T17:43:52.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.175+0000 7fa7c659d640 1 -- start start 2024-09-13T17:43:52.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.175+0000 7fa7c659d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa7c0096a70 0x7fa7c013c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:52.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.175+0000 7fa7c659d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa7c012c900 0x7fa7c013ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:52.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.175+0000 7fa7c659d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7c012ccd0 0x7fa7c01161e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:52.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.175+0000 7fa7c659d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c006cbf0 con 0x7fa7c012ccd0 2024-09-13T17:43:52.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.175+0000 7fa7c659d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c01306d0 con 0x7fa7c0096a70 2024-09-13T17:43:52.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.175+0000 7fa7c659d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c01198f0 con 0x7fa7c012c900 2024-09-13T17:43:52.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.175+0000 7fa7bffff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa7c0096a70 0x7fa7c013c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:52.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.175+0000 7fa7b7fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa7c012c900 0x7fa7c013ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:52.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.175+0000 7fa7bffff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa7c0096a70 0x7fa7c013c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:38554/0 (socket says 172.21.15.102:38554) 2024-09-13T17:43:52.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.175+0000 7fa7c4b3a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7c012ccd0 0x7fa7c01161e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:52.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.175+0000 7fa7bffff640 1 -- 172.21.15.102:0/77851680 learned_addr learned my addr 172.21.15.102:0/77851680 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:52.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.175+0000 7fa7bffff640 1 -- 172.21.15.102:0/77851680 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa7c012c900 msgr2=0x7fa7c013ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:52.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.175+0000 7fa7bffff640 1 --2- 172.21.15.102:0/77851680 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa7c012c900 0x7fa7c013ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.175+0000 7fa7bffff640 1 -- 172.21.15.102:0/77851680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7c012ccd0 msgr2=0x7fa7c01161e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:52.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.175+0000 7fa7bffff640 1 --2- 172.21.15.102:0/77851680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7c012ccd0 0x7fa7c01161e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.175+0000 7fa7bffff640 1 -- 172.21.15.102:0/77851680 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b8009a20 con 0x7fa7c0096a70 2024-09-13T17:43:52.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.175+0000 7fa7b7fff640 1 --2- 172.21.15.102:0/77851680 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa7c012c900 0x7fa7c013ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:52.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.176+0000 7fa7bffff640 1 --2- 172.21.15.102:0/77851680 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa7c0096a70 0x7fa7c013c950 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0009360 tx=0x7fa7b0032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:52.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.176+0000 7fa7bcff9640 1 -- 172.21.15.102:0/77851680 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0031700 con 0x7fa7c0096a70 2024-09-13T17:43:52.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.176+0000 7fa7bcff9640 1 -- 172.21.15.102:0/77851680 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b0031860 con 0x7fa7c0096a70 2024-09-13T17:43:52.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.176+0000 7fa7c659d640 1 -- 172.21.15.102:0/77851680 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7b0009390 con 0x7fa7c0096a70 2024-09-13T17:43:52.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.176+0000 7fa7bcff9640 1 -- 172.21.15.102:0/77851680 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0031c00 con 0x7fa7c0096a70 2024-09-13T17:43:52.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.176+0000 7fa7c659d640 1 -- 172.21.15.102:0/77851680 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c0117cc0 con 0x7fa7c0096a70 2024-09-13T17:43:52.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.177+0000 7fa7bcff9640 1 -- 172.21.15.102:0/77851680 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fa7b003c400 con 0x7fa7c0096a70 2024-09-13T17:43:52.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.177+0000 7fa7bcff9640 1 -- 172.21.15.102:0/77851680 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7fa7b0062680 con 0x7fa7c0096a70 2024-09-13T17:43:52.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.177+0000 7fa7c659d640 1 -- 172.21.15.102:0/77851680 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7c0064000 con 0x7fa7c0096a70 2024-09-13T17:43:52.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.181+0000 7fa7bcff9640 1 -- 172.21.15.102:0/77851680 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fa7b0049080 con 0x7fa7c0096a70 2024-09-13T17:43:52.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.333+0000 7fa7c659d640 1 -- 172.21.15.102:0/77851680 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa7c0119590 con 0x7fa7c0096a70 2024-09-13T17:43:52.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.334+0000 7fa7bcff9640 1 -- 172.21.15.102:0/77851680 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa7b0037070 con 0x7fa7c0096a70 2024-09-13T17:43:52.336 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-09-13T17:43:52.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.335+0000 7fa7c659d640 1 -- 172.21.15.102:0/77851680 >> v1:172.21.15.102:6816/34937 conn(0x7fa7a0031160 legacy=0x7fa7a0033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:52.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.335+0000 7fa7c659d640 1 -- 172.21.15.102:0/77851680 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa7c0096a70 msgr2=0x7fa7c013c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:52.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.335+0000 7fa7c659d640 1 --2- 172.21.15.102:0/77851680 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa7c0096a70 0x7fa7c013c950 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0009360 tx=0x7fa7b0032000 comp rx=0 tx=0).stop 2024-09-13T17:43:52.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.335+0000 7fa7c659d640 1 -- 172.21.15.102:0/77851680 shutdown_connections 2024-09-13T17:43:52.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.335+0000 7fa7c659d640 1 --2- 172.21.15.102:0/77851680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7c012ccd0 0x7fa7c01161e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.335+0000 7fa7c659d640 1 --2- 172.21.15.102:0/77851680 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa7c012c900 0x7fa7c013ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.335+0000 7fa7c659d640 1 --2- 172.21.15.102:0/77851680 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa7c0096a70 0x7fa7c013c950 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.335+0000 7fa7c659d640 1 -- 172.21.15.102:0/77851680 >> 172.21.15.102:0/77851680 conn(0x7fa7c009e6b0 msgr2=0x7fa7c00972e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:52.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.335+0000 7fa7c659d640 1 -- 172.21.15.102:0/77851680 shutdown_connections 2024-09-13T17:43:52.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.335+0000 7fa7c659d640 1 -- 172.21.15.102:0/77851680 wait complete. 2024-09-13T17:43:52.348 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:43:52.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.442+0000 7f5137d9d640 1 Processor -- start 2024-09-13T17:43:52.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.442+0000 7f5137d9d640 1 -- start start 2024-09-13T17:43:52.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.443+0000 7f5137d9d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5130097d30 0x7f5130132610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:52.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.443+0000 7f5137d9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5130132b50 0x7f5130134f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:52.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.443+0000 7f5137d9d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f513006a850 con 0x7f5130096df0 2024-09-13T17:43:52.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.443+0000 7f5137d9d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5130098290 con 0x7f5130097d30 2024-09-13T17:43:52.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.443+0000 7f5137d9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5130096bf0 con 0x7f5130132b50 2024-09-13T17:43:52.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.443+0000 7f5135b39640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f5130096df0 0x7f51300971c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:58876/0 (socket says 172.21.15.102:58876) 2024-09-13T17:43:52.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.443+0000 7f5135b39640 1 -- 172.21.15.102:0/3965972790 learned_addr learned my addr 172.21.15.102:0/3965972790 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:52.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.443+0000 7f5135338640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5130097d30 0x7f5130132610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:52.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.443+0000 7f513633a640 1 --2- 172.21.15.102:0/3965972790 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5130132b50 0x7f5130134f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:52.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.443+0000 7f5135338640 1 -- 172.21.15.102:0/3965972790 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5130132b50 msgr2=0x7f5130134f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:52.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.443+0000 7f5135338640 1 --2- 172.21.15.102:0/3965972790 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5130132b50 0x7f5130134f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.443+0000 7f5135338640 1 -- 172.21.15.102:0/3965972790 >> v1:172.21.15.102:6789/0 conn(0x7f5130096df0 legacy=0x7f51300971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:52.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.443+0000 7f5135338640 1 -- 172.21.15.102:0/3965972790 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f513006a590 con 0x7f5130097d30 2024-09-13T17:43:52.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.443+0000 7f5134b37640 1 -- 172.21.15.102:0/3965972790 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1062381462 0 0) 0x7f513006a850 con 0x7f5130096df0 2024-09-13T17:43:52.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.443+0000 7f513633a640 1 --2- 172.21.15.102:0/3965972790 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5130132b50 0x7f5130134f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:52.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.444+0000 7f5135338640 1 --2- 172.21.15.102:0/3965972790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5130097d30 0x7f5130132610 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5120009880 tx=0x7f51200090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55d38d4410a4437 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:52.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.444+0000 7f5134b37640 1 -- 172.21.15.102:0/3965972790 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512002ace0 con 0x7f5130097d30 2024-09-13T17:43:52.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.444+0000 7f5134b37640 1 -- 172.21.15.102:0/3965972790 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512002ae40 con 0x7f5130097d30 2024-09-13T17:43:52.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.444+0000 7f5134b37640 1 -- 172.21.15.102:0/3965972790 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5120032670 con 0x7f5130097d30 2024-09-13T17:43:52.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.444+0000 7f5137d9d640 1 -- 172.21.15.102:0/3965972790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5130097d30 msgr2=0x7f5130132610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:52.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.444+0000 7f5137d9d640 1 --2- 172.21.15.102:0/3965972790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5130097d30 0x7f5130132610 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5120009880 tx=0x7f51200090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.444+0000 7f5137d9d640 1 -- 172.21.15.102:0/3965972790 shutdown_connections 2024-09-13T17:43:52.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.444+0000 7f5137d9d640 1 --2- 172.21.15.102:0/3965972790 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5130132b50 0x7f5130134f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.444+0000 7f5137d9d640 1 --2- 172.21.15.102:0/3965972790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5130097d30 0x7f5130132610 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.444+0000 7f5137d9d640 1 -- 172.21.15.102:0/3965972790 >> 172.21.15.102:0/3965972790 conn(0x7f513009e720 msgr2=0x7f513009eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:52.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.444+0000 7f5137d9d640 1 -- 172.21.15.102:0/3965972790 shutdown_connections 2024-09-13T17:43:52.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.444+0000 7f5137d9d640 1 -- 172.21.15.102:0/3965972790 wait complete. 2024-09-13T17:43:52.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.445+0000 7f5137d9d640 1 Processor -- start 2024-09-13T17:43:52.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.445+0000 7f5137d9d640 1 -- start start 2024-09-13T17:43:52.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.446+0000 7f5137d9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5130096df0 0x7f51301300a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:52.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.446+0000 7f5137d9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5130097d30 0x7f51301305e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:52.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.446+0000 7f5137d9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5130132b50 0x7f5130130b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:52.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.446+0000 7f5137d9d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5130116180 con 0x7f5130096df0 2024-09-13T17:43:52.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.446+0000 7f5137d9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513012a970 con 0x7f5130097d30 2024-09-13T17:43:52.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.446+0000 7f5137d9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f51301321b0 con 0x7f5130132b50 2024-09-13T17:43:52.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.446+0000 7f513633a640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5130132b50 0x7f5130130b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:52.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.446+0000 7f5135338640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5130097d30 0x7f51301305e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:52.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.446+0000 7f5135b39640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5130096df0 0x7f51301300a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:52.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.446+0000 7f5135338640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5130097d30 0x7f51301305e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:38564/0 (socket says 172.21.15.102:38564) 2024-09-13T17:43:52.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.446+0000 7f513633a640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5130132b50 0x7f5130130b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:36178/0 (socket says 172.21.15.102:36178) 2024-09-13T17:43:52.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.446+0000 7f5135338640 1 -- 172.21.15.102:0/2685991991 learned_addr learned my addr 172.21.15.102:0/2685991991 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:52.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.446+0000 7f513633a640 1 -- 172.21.15.102:0/2685991991 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5130097d30 msgr2=0x7f51301305e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:52.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.446+0000 7f513633a640 1 --2- 172.21.15.102:0/2685991991 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5130097d30 0x7f51301305e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.446+0000 7f513633a640 1 -- 172.21.15.102:0/2685991991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5130096df0 msgr2=0x7f51301300a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:52.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.446+0000 7f513633a640 1 --2- 172.21.15.102:0/2685991991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5130096df0 0x7f51301300a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.446+0000 7f513633a640 1 -- 172.21.15.102:0/2685991991 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512800de30 con 0x7f5130132b50 2024-09-13T17:43:52.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.446+0000 7f5135b39640 1 --2- 172.21.15.102:0/2685991991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5130096df0 0x7f51301300a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:52.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.446+0000 7f513633a640 1 --2- 172.21.15.102:0/2685991991 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5130132b50 0x7f5130130b20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5128012290 tx=0x7f512800db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:52.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.446+0000 7f5125ffb640 1 -- 172.21.15.102:0/2685991991 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5128012e60 con 0x7f5130132b50 2024-09-13T17:43:52.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.446+0000 7f5125ffb640 1 -- 172.21.15.102:0/2685991991 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5128013040 con 0x7f5130132b50 2024-09-13T17:43:52.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.446+0000 7f5125ffb640 1 -- 172.21.15.102:0/2685991991 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5128019850 con 0x7f5130132b50 2024-09-13T17:43:52.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.446+0000 7f5137d9d640 1 -- 172.21.15.102:0/2685991991 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5120009390 con 0x7f5130132b50 2024-09-13T17:43:52.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.447+0000 7f5137d9d640 1 -- 172.21.15.102:0/2685991991 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5130131c70 con 0x7f5130132b50 2024-09-13T17:43:52.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.447+0000 7f5125ffb640 1 -- 172.21.15.102:0/2685991991 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f5128019c40 con 0x7f5130132b50 2024-09-13T17:43:52.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.447+0000 7f5125ffb640 1 -- 172.21.15.102:0/2685991991 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f51280471e0 con 0x7f5130132b50 2024-09-13T17:43:52.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.448+0000 7f5137d9d640 1 -- 172.21.15.102:0/2685991991 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51301162c0 con 0x7f5130132b50 2024-09-13T17:43:52.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.452+0000 7f5125ffb640 1 -- 172.21.15.102:0/2685991991 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f51301162c0 con 0x7f5130132b50 2024-09-13T17:43:52.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.610+0000 7f5137d9d640 1 -- 172.21.15.102:0/2685991991 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f51301162c0 con 0x7f5130132b50 2024-09-13T17:43:52.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.610+0000 7f5125ffb640 1 -- 172.21.15.102:0/2685991991 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f51301162c0 con 0x7f5130132b50 2024-09-13T17:43:52.612 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-09-13T17:43:52.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.612+0000 7f5137d9d640 1 -- 172.21.15.102:0/2685991991 >> v1:172.21.15.102:6816/34937 conn(0x7f5114031190 legacy=0x7f5114033640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:52.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.612+0000 7f5137d9d640 1 -- 172.21.15.102:0/2685991991 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5130132b50 msgr2=0x7f5130130b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:52.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.612+0000 7f5137d9d640 1 --2- 172.21.15.102:0/2685991991 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5130132b50 0x7f5130130b20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5128012290 tx=0x7f512800db90 comp rx=0 tx=0).stop 2024-09-13T17:43:52.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.612+0000 7f5137d9d640 1 -- 172.21.15.102:0/2685991991 shutdown_connections 2024-09-13T17:43:52.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.612+0000 7f5137d9d640 1 --2- 172.21.15.102:0/2685991991 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5130132b50 0x7f5130130b20 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.612+0000 7f5137d9d640 1 --2- 172.21.15.102:0/2685991991 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5130097d30 0x7f51301305e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.612+0000 7f5137d9d640 1 --2- 172.21.15.102:0/2685991991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5130096df0 0x7f51301300a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.612+0000 7f5137d9d640 1 -- 172.21.15.102:0/2685991991 >> 172.21.15.102:0/2685991991 conn(0x7f513009e720 msgr2=0x7f513006df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:52.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.612+0000 7f5137d9d640 1 -- 172.21.15.102:0/2685991991 shutdown_connections 2024-09-13T17:43:52.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.612+0000 7f5137d9d640 1 -- 172.21.15.102:0/2685991991 wait complete. 2024-09-13T17:43:52.625 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:43:52.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.717+0000 7fdff179d640 1 Processor -- start 2024-09-13T17:43:52.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.717+0000 7fdff179d640 1 -- start start 2024-09-13T17:43:52.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.718+0000 7fdff179d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fdfec068920 0x7fdfec09d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:52.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.718+0000 7fdff179d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdfec09e050 0x7fdfec12fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:52.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.718+0000 7fdff179d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfec0695a0 con 0x7fdfec09dc50 2024-09-13T17:43:52.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.718+0000 7fdff179d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfec068cf0 con 0x7fdfec068920 2024-09-13T17:43:52.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.718+0000 7fdff179d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec09e4a0 con 0x7fdfec09e050 2024-09-13T17:43:52.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.718+0000 7fdfeaffd640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fdfec068920 0x7fdfec09d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:52.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.718+0000 7fdfea7fc640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fdfec09dc50 0x7fdfec12f310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:58882/0 (socket says 172.21.15.102:58882) 2024-09-13T17:43:52.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.718+0000 7fdfea7fc640 1 -- 172.21.15.102:0/3175610235 learned_addr learned my addr 172.21.15.102:0/3175610235 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:52.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.718+0000 7fdfeb7fe640 1 --2- 172.21.15.102:0/3175610235 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdfec09e050 0x7fdfec12fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:52.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.718+0000 7fdfeb7fe640 1 -- 172.21.15.102:0/3175610235 >> v2:172.21.15.102:3300/0 conn(0x7fdfec068920 msgr2=0x7fdfec09d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:52.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.718+0000 7fdfeb7fe640 1 --2- 172.21.15.102:0/3175610235 >> v2:172.21.15.102:3300/0 conn(0x7fdfec068920 0x7fdfec09d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.718+0000 7fdfeb7fe640 1 -- 172.21.15.102:0/3175610235 >> v1:172.21.15.102:6789/0 conn(0x7fdfec09dc50 legacy=0x7fdfec12f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:52.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.718+0000 7fdfeb7fe640 1 -- 172.21.15.102:0/3175610235 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfec09f090 con 0x7fdfec09e050 2024-09-13T17:43:52.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.718+0000 7fdfeaffd640 1 --2- 172.21.15.102:0/3175610235 >> v2:172.21.15.102:3300/0 conn(0x7fdfec068920 0x7fdfec09d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:52.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.719+0000 7fdfeb7fe640 1 --2- 172.21.15.102:0/3175610235 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdfec09e050 0x7fdfec12fa20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4009e60 tx=0x7fdfe4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb875e849a82810f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:52.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.719+0000 7fdfe9ffb640 1 -- 172.21.15.102:0/3175610235 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4007550 con 0x7fdfec09e050 2024-09-13T17:43:52.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.719+0000 7fdfe9ffb640 1 -- 172.21.15.102:0/3175610235 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe402d050 con 0x7fdfec09e050 2024-09-13T17:43:52.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.719+0000 7fdfe9ffb640 1 -- 172.21.15.102:0/3175610235 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4007920 con 0x7fdfec09e050 2024-09-13T17:43:52.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.720+0000 7fdff179d640 1 -- 172.21.15.102:0/3175610235 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdfec09e050 msgr2=0x7fdfec12fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:52.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.720+0000 7fdff179d640 1 --2- 172.21.15.102:0/3175610235 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdfec09e050 0x7fdfec12fa20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4009e60 tx=0x7fdfe4009780 comp rx=0 tx=0).stop 2024-09-13T17:43:52.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.720+0000 7fdff179d640 1 -- 172.21.15.102:0/3175610235 shutdown_connections 2024-09-13T17:43:52.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.720+0000 7fdff179d640 1 --2- 172.21.15.102:0/3175610235 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdfec09e050 0x7fdfec12fa20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.720+0000 7fdff179d640 1 --2- 172.21.15.102:0/3175610235 >> v2:172.21.15.102:3300/0 conn(0x7fdfec068920 0x7fdfec09d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.720+0000 7fdff179d640 1 -- 172.21.15.102:0/3175610235 >> 172.21.15.102:0/3175610235 conn(0x7fdfec06f320 msgr2=0x7fdfec0a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:52.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.720+0000 7fdff179d640 1 -- 172.21.15.102:0/3175610235 shutdown_connections 2024-09-13T17:43:52.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.720+0000 7fdff179d640 1 -- 172.21.15.102:0/3175610235 wait complete. 2024-09-13T17:43:52.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.721+0000 7fdff179d640 1 Processor -- start 2024-09-13T17:43:52.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.721+0000 7fdff179d640 1 -- start start 2024-09-13T17:43:52.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.721+0000 7fdff179d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdfec068920 0x7fdfec11c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:52.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.721+0000 7fdff179d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfec09dc50 0x7fdfec11c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:52.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.721+0000 7fdff179d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdfec09e050 0x7fdfec13eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:52.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.721+0000 7fdff179d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec11ed10 con 0x7fdfec09dc50 2024-09-13T17:43:52.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.721+0000 7fdff179d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec11d8e0 con 0x7fdfec068920 2024-09-13T17:43:52.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.721+0000 7fdff179d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec09e620 con 0x7fdfec09e050 2024-09-13T17:43:52.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.721+0000 7fdfea7fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfec09dc50 0x7fdfec11c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:52.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.721+0000 7fdfeb7fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdfec09e050 0x7fdfec13eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:52.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.721+0000 7fdfea7fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfec09dc50 0x7fdfec11c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:46212/0 (socket says 172.21.15.102:46212) 2024-09-13T17:43:52.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.721+0000 7fdfea7fc640 1 -- 172.21.15.102:0/964092061 learned_addr learned my addr 172.21.15.102:0/964092061 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:52.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.721+0000 7fdfeaffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdfec068920 0x7fdfec11c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:52.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.721+0000 7fdfea7fc640 1 -- 172.21.15.102:0/964092061 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdfec09e050 msgr2=0x7fdfec13eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:52.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.721+0000 7fdfea7fc640 1 --2- 172.21.15.102:0/964092061 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdfec09e050 0x7fdfec13eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.721+0000 7fdfea7fc640 1 -- 172.21.15.102:0/964092061 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdfec068920 msgr2=0x7fdfec11c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:52.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.721+0000 7fdfea7fc640 1 --2- 172.21.15.102:0/964092061 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdfec068920 0x7fdfec11c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.721+0000 7fdfea7fc640 1 -- 172.21.15.102:0/964092061 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfdc009390 con 0x7fdfec09dc50 2024-09-13T17:43:52.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.721+0000 7fdfeb7fe640 1 --2- 172.21.15.102:0/964092061 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdfec09e050 0x7fdfec13eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:52.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.721+0000 7fdfeaffd640 1 --2- 172.21.15.102:0/964092061 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdfec068920 0x7fdfec11c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:52.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.722+0000 7fdfea7fc640 1 --2- 172.21.15.102:0/964092061 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfec09dc50 0x7fdfec11c6c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000c590 tx=0x7fdfe0007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:52.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.722+0000 7fdfd2ffd640 1 -- 172.21.15.102:0/964092061 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe0005aa0 con 0x7fdfec09dc50 2024-09-13T17:43:52.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.722+0000 7fdfd2ffd640 1 -- 172.21.15.102:0/964092061 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe0005c20 con 0x7fdfec09dc50 2024-09-13T17:43:52.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.722+0000 7fdff179d640 1 -- 172.21.15.102:0/964092061 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfe4009a20 con 0x7fdfec09dc50 2024-09-13T17:43:52.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.722+0000 7fdfd2ffd640 1 -- 172.21.15.102:0/964092061 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe00070f0 con 0x7fdfec09dc50 2024-09-13T17:43:52.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.722+0000 7fdff179d640 1 -- 172.21.15.102:0/964092061 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfec11fa00 con 0x7fdfec09dc50 2024-09-13T17:43:52.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.722+0000 7fdfd2ffd640 1 -- 172.21.15.102:0/964092061 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fdfe0007540 con 0x7fdfec09dc50 2024-09-13T17:43:52.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.723+0000 7fdfd2ffd640 1 -- 172.21.15.102:0/964092061 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7fdfe0041490 con 0x7fdfec09dc50 2024-09-13T17:43:52.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.723+0000 7fdff179d640 1 -- 172.21.15.102:0/964092061 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfec064000 con 0x7fdfec09dc50 2024-09-13T17:43:52.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.727+0000 7fdfd2ffd640 1 -- 172.21.15.102:0/964092061 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fdfe0018020 con 0x7fdfec09dc50 2024-09-13T17:43:52.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.888+0000 7fdff179d640 1 -- 172.21.15.102:0/964092061 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdfec11e930 con 0x7fdfec09dc50 2024-09-13T17:43:52.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.889+0000 7fdfd2ffd640 1 -- 172.21.15.102:0/964092061 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7fdfe0010d70 con 0x7fdfec09dc50 2024-09-13T17:43:52.890 INFO:teuthology.orchestra.run.smithi102.stderr:listed 0 entries 2024-09-13T17:43:52.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.890+0000 7fdff179d640 1 -- 172.21.15.102:0/964092061 >> v1:172.21.15.102:6816/34937 conn(0x7fdfd8031160 legacy=0x7fdfd8033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:52.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.890+0000 7fdff179d640 1 -- 172.21.15.102:0/964092061 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfec09dc50 msgr2=0x7fdfec11c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:52.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.890+0000 7fdff179d640 1 --2- 172.21.15.102:0/964092061 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfec09dc50 0x7fdfec11c6c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000c590 tx=0x7fdfe0007ee0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.890+0000 7fdff179d640 1 -- 172.21.15.102:0/964092061 shutdown_connections 2024-09-13T17:43:52.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.890+0000 7fdff179d640 1 --2- 172.21.15.102:0/964092061 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdfec09e050 0x7fdfec13eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.890+0000 7fdff179d640 1 --2- 172.21.15.102:0/964092061 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfec09dc50 0x7fdfec11c6c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.890+0000 7fdff179d640 1 --2- 172.21.15.102:0/964092061 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdfec068920 0x7fdfec11c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.890+0000 7fdff179d640 1 -- 172.21.15.102:0/964092061 >> 172.21.15.102:0/964092061 conn(0x7fdfec06f320 msgr2=0x7fdfec098560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:52.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.890+0000 7fdff179d640 1 -- 172.21.15.102:0/964092061 shutdown_connections 2024-09-13T17:43:52.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:52.890+0000 7fdff179d640 1 -- 172.21.15.102:0/964092061 wait complete. 2024-09-13T17:43:52.903 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T17:43:52.903 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:43:52.922 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:43:53.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.015+0000 7f06d5d9d640 1 Processor -- start 2024-09-13T17:43:53.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.015+0000 7f06d5d9d640 1 -- start start 2024-09-13T17:43:53.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.015+0000 7f06d5d9d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f06d0128330 0x7f06d0128700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:53.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.015+0000 7f06d5d9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f06d0128c40 0x7f06d012b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:53.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.015+0000 7f06d5d9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f06d012bd80 0x7f06d012c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:53.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.015+0000 7f06d5d9d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06d012c710 con 0x7f06d0128330 2024-09-13T17:43:53.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.015+0000 7f06d5d9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d0097be0 con 0x7f06d012bd80 2024-09-13T17:43:53.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.015+0000 7f06d5d9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f06d0098e70 con 0x7f06d0128c40 2024-09-13T17:43:53.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.015+0000 7f06c7fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f06d0128c40 0x7f06d012b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:53.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.015+0000 7f06cf7fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f06d0128330 0x7f06d0128700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:53.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.015+0000 7f06cffff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f06d012bd80 0x7f06d012c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:53.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.015+0000 7f06c7fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f06d0128c40 0x7f06d012b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:36206/0 (socket says 172.21.15.102:36206) 2024-09-13T17:43:53.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.015+0000 7f06cffff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f06d012bd80 0x7f06d012c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:38590/0 (socket says 172.21.15.102:38590) 2024-09-13T17:43:53.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.015+0000 7f06cf7fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f06d0128330 0x7f06d0128700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:46228/0 (socket says 172.21.15.102:46228) 2024-09-13T17:43:53.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.015+0000 7f06c7fff640 1 -- 172.21.15.102:0/2100566282 learned_addr learned my addr 172.21.15.102:0/2100566282 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:53.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.016+0000 7f06c7fff640 1 -- 172.21.15.102:0/2100566282 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f06d012bd80 msgr2=0x7f06d012c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:53.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.016+0000 7f06c7fff640 1 --2- 172.21.15.102:0/2100566282 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f06d012bd80 0x7f06d012c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.016+0000 7f06c7fff640 1 -- 172.21.15.102:0/2100566282 >> v2:172.21.15.102:3300/0 conn(0x7f06d0128330 msgr2=0x7f06d0128700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:53.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.016+0000 7f06c7fff640 1 --2- 172.21.15.102:0/2100566282 >> v2:172.21.15.102:3300/0 conn(0x7f06d0128330 0x7f06d0128700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.016+0000 7f06c7fff640 1 -- 172.21.15.102:0/2100566282 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d0098310 con 0x7f06d0128c40 2024-09-13T17:43:53.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.016+0000 7f06cf7fe640 1 --2- 172.21.15.102:0/2100566282 >> v2:172.21.15.102:3300/0 conn(0x7f06d0128330 0x7f06d0128700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:53.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.016+0000 7f06cffff640 1 --2- 172.21.15.102:0/2100566282 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f06d012bd80 0x7f06d012c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:53.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.016+0000 7f06c7fff640 1 --2- 172.21.15.102:0/2100566282 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f06d0128c40 0x7f06d012b840 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f06b800bbc0 tx=0x7f06b800b460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d1deb82da2bd50a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:53.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.017+0000 7f06ceffd640 1 -- 172.21.15.102:0/2100566282 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b8032620 con 0x7f06d0128c40 2024-09-13T17:43:53.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.017+0000 7f06ceffd640 1 -- 172.21.15.102:0/2100566282 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b802e040 con 0x7f06d0128c40 2024-09-13T17:43:53.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.017+0000 7f06ceffd640 1 -- 172.21.15.102:0/2100566282 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b80329f0 con 0x7f06d0128c40 2024-09-13T17:43:53.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.017+0000 7f06d5d9d640 1 -- 172.21.15.102:0/2100566282 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f06d0128c40 msgr2=0x7f06d012b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:53.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.017+0000 7f06d5d9d640 1 --2- 172.21.15.102:0/2100566282 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f06d0128c40 0x7f06d012b840 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f06b800bbc0 tx=0x7f06b800b460 comp rx=0 tx=0).stop 2024-09-13T17:43:53.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.017+0000 7f06d5d9d640 1 -- 172.21.15.102:0/2100566282 shutdown_connections 2024-09-13T17:43:53.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.017+0000 7f06d5d9d640 1 --2- 172.21.15.102:0/2100566282 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f06d012bd80 0x7f06d012c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.017+0000 7f06d5d9d640 1 --2- 172.21.15.102:0/2100566282 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f06d0128c40 0x7f06d012b840 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.017+0000 7f06d5d9d640 1 --2- 172.21.15.102:0/2100566282 >> v2:172.21.15.102:3300/0 conn(0x7f06d0128330 0x7f06d0128700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.017+0000 7f06d5d9d640 1 -- 172.21.15.102:0/2100566282 >> 172.21.15.102:0/2100566282 conn(0x7f06d009e820 msgr2=0x7f06d009ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:53.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.017+0000 7f06d5d9d640 1 -- 172.21.15.102:0/2100566282 shutdown_connections 2024-09-13T17:43:53.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.017+0000 7f06d5d9d640 1 -- 172.21.15.102:0/2100566282 wait complete. 2024-09-13T17:43:53.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.018+0000 7f06d5d9d640 1 Processor -- start 2024-09-13T17:43:53.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.018+0000 7f06d5d9d640 1 -- start start 2024-09-13T17:43:53.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.018+0000 7f06d5d9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06d0128330 0x7f06d0114e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:53.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.018+0000 7f06d5d9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f06d0128c40 0x7f06d01153c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:53.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.018+0000 7f06d5d9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f06d012bd80 0x7f06d0115900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:53.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.018+0000 7f06d5d9d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d006ebb0 con 0x7f06d0128330 2024-09-13T17:43:53.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.018+0000 7f06d5d9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d012ba90 con 0x7f06d012bd80 2024-09-13T17:43:53.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.018+0000 7f06d5d9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f06d0125f80 con 0x7f06d0128c40 2024-09-13T17:43:53.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.019+0000 7f06cf7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06d0128330 0x7f06d0114e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:53.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.019+0000 7f06c7fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f06d0128c40 0x7f06d01153c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:53.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.019+0000 7f06cffff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f06d012bd80 0x7f06d0115900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:53.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.019+0000 7f06cf7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06d0128330 0x7f06d0114e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:46234/0 (socket says 172.21.15.102:46234) 2024-09-13T17:43:53.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.019+0000 7f06cffff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f06d012bd80 0x7f06d0115900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:38596/0 (socket says 172.21.15.102:38596) 2024-09-13T17:43:53.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.019+0000 7f06cf7fe640 1 -- 172.21.15.102:0/3847050278 learned_addr learned my addr 172.21.15.102:0/3847050278 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:53.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.019+0000 7f06c7fff640 1 -- 172.21.15.102:0/3847050278 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f06d012bd80 msgr2=0x7f06d0115900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:53.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.019+0000 7f06c7fff640 1 --2- 172.21.15.102:0/3847050278 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f06d012bd80 0x7f06d0115900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.019+0000 7f06c7fff640 1 -- 172.21.15.102:0/3847050278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06d0128330 msgr2=0x7f06d0114e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:53.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.019+0000 7f06c7fff640 1 --2- 172.21.15.102:0/3847050278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06d0128330 0x7f06d0114e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.019+0000 7f06c7fff640 1 -- 172.21.15.102:0/3847050278 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c0009390 con 0x7f06d0128c40 2024-09-13T17:43:53.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.019+0000 7f06cf7fe640 1 --2- 172.21.15.102:0/3847050278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06d0128330 0x7f06d0114e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:53.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.019+0000 7f06c7fff640 1 --2- 172.21.15.102:0/3847050278 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f06d0128c40 0x7f06d01153c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f06b8034000 tx=0x7f06b803a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:53.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.019+0000 7f06c77fe640 1 -- 172.21.15.102:0/3847050278 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b8041070 con 0x7f06d0128c40 2024-09-13T17:43:53.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.019+0000 7f06c77fe640 1 -- 172.21.15.102:0/3847050278 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b8032e20 con 0x7f06d0128c40 2024-09-13T17:43:53.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.019+0000 7f06d5d9d640 1 -- 172.21.15.102:0/3847050278 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c8009a20 con 0x7f06d0128c40 2024-09-13T17:43:53.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.019+0000 7f06c77fe640 1 -- 172.21.15.102:0/3847050278 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b803cb30 con 0x7f06d0128c40 2024-09-13T17:43:53.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.019+0000 7f06d5d9d640 1 -- 172.21.15.102:0/3847050278 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b800b700 con 0x7f06d0128c40 2024-09-13T17:43:53.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.020+0000 7f06c77fe640 1 -- 172.21.15.102:0/3847050278 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f06b8045070 con 0x7f06d0128c40 2024-09-13T17:43:53.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.020+0000 7f06c77fe640 1 -- 172.21.15.102:0/3847050278 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f06b8064eb0 con 0x7f06d0128c40 2024-09-13T17:43:53.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.020+0000 7f06d5d9d640 1 -- 172.21.15.102:0/3847050278 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06d0117060 con 0x7f06d0128c40 2024-09-13T17:43:53.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.024+0000 7f06c77fe640 1 -- 172.21.15.102:0/3847050278 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f06d0117060 con 0x7f06d0128c40 2024-09-13T17:43:53.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.176+0000 7f06d5d9d640 1 -- 172.21.15.102:0/3847050278 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06d0117060 con 0x7f06d0128c40 2024-09-13T17:43:53.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.177+0000 7f06c77fe640 1 -- 172.21.15.102:0/3847050278 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+11660 (secure 0 0 0) 0x7f06d0117060 con 0x7f06d0128c40 2024-09-13T17:43:53.179 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:43:53.179 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":23,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:42:48.385896+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:04.672265+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:05.583134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.17","mappings":[{"from":3,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]},{"pgid":"4.25","mappings":[{"from":3,"to":2}]},{"pgid":"4.33","mappings":[{"from":3,"to":0}]},{"pgid":"4.3a","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:43:53.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.179+0000 7f06d5d9d640 1 -- 172.21.15.102:0/3847050278 >> v1:172.21.15.102:6816/34937 conn(0x7f06b0031160 legacy=0x7f06b0033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:53.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.179+0000 7f06d5d9d640 1 -- 172.21.15.102:0/3847050278 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f06d0128c40 msgr2=0x7f06d01153c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:53.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.179+0000 7f06d5d9d640 1 --2- 172.21.15.102:0/3847050278 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f06d0128c40 0x7f06d01153c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f06b8034000 tx=0x7f06b803a000 comp rx=0 tx=0).stop 2024-09-13T17:43:53.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.179+0000 7f06d5d9d640 1 -- 172.21.15.102:0/3847050278 shutdown_connections 2024-09-13T17:43:53.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.179+0000 7f06d5d9d640 1 --2- 172.21.15.102:0/3847050278 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f06d012bd80 0x7f06d0115900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.179+0000 7f06d5d9d640 1 --2- 172.21.15.102:0/3847050278 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f06d0128c40 0x7f06d01153c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.179+0000 7f06d5d9d640 1 --2- 172.21.15.102:0/3847050278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06d0128330 0x7f06d0114e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.179+0000 7f06d5d9d640 1 -- 172.21.15.102:0/3847050278 >> 172.21.15.102:0/3847050278 conn(0x7f06d009e820 msgr2=0x7f06d012aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:53.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.179+0000 7f06d5d9d640 1 -- 172.21.15.102:0/3847050278 shutdown_connections 2024-09-13T17:43:53.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.179+0000 7f06d5d9d640 1 -- 172.21.15.102:0/3847050278 wait complete. 2024-09-13T17:43:53.192 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:54.442612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T17:41:59.076782+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 16, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:42:04.672265+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:42:05.583134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:43:53.192 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:43:53.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.286+0000 7fc1b551d640 1 Processor -- start 2024-09-13T17:43:53.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.286+0000 7fc1b551d640 1 -- start start 2024-09-13T17:43:53.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.287+0000 7fc1b551d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc1b00982e0 0x7fc1b009e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:53.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.287+0000 7fc1b551d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1b00990b0 0x7fc1b00994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:53.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.287+0000 7fc1b551d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1b0099a30 con 0x7fc1b009e5d0 2024-09-13T17:43:53.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.287+0000 7fc1b551d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b009f620 con 0x7fc1b00982e0 2024-09-13T17:43:53.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.287+0000 7fc1b551d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b00986b0 con 0x7fc1b00990b0 2024-09-13T17:43:53.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.287+0000 7fc1aeffd640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc1b00982e0 0x7fc1b009e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:53.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.287+0000 7fc1af7fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1b00990b0 0x7fc1b00994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:53.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.287+0000 7fc1ae7fc640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fc1b009e5d0 0x7fc1b00989a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:58884/0 (socket says 172.21.15.102:58884) 2024-09-13T17:43:53.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.287+0000 7fc1ae7fc640 1 -- 172.21.15.102:0/4288169766 learned_addr learned my addr 172.21.15.102:0/4288169766 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:53.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.287+0000 7fc1af7fe640 1 -- 172.21.15.102:0/4288169766 >> v2:172.21.15.102:3300/0 conn(0x7fc1b00982e0 msgr2=0x7fc1b009e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:53.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.287+0000 7fc1af7fe640 1 --2- 172.21.15.102:0/4288169766 >> v2:172.21.15.102:3300/0 conn(0x7fc1b00982e0 0x7fc1b009e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.287+0000 7fc1af7fe640 1 -- 172.21.15.102:0/4288169766 >> v1:172.21.15.102:6789/0 conn(0x7fc1b009e5d0 legacy=0x7fc1b00989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:53.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.287+0000 7fc1adffb640 1 -- 172.21.15.102:0/4288169766 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2071141067 0 0) 0x7fc198001110 con 0x7fc1b009e5d0 2024-09-13T17:43:53.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.287+0000 7fc1af7fe640 1 -- 172.21.15.102:0/4288169766 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b0097820 con 0x7fc1b00990b0 2024-09-13T17:43:53.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.287+0000 7fc1aeffd640 1 --2- 172.21.15.102:0/4288169766 >> v2:172.21.15.102:3300/0 conn(0x7fc1b00982e0 0x7fc1b009e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:53.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.288+0000 7fc1af7fe640 1 --2- 172.21.15.102:0/4288169766 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1b00990b0 0x7fc1b00994f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8009880 tx=0x7fc1a80090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=569e7617203d4718 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:53.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.288+0000 7fc1adffb640 1 -- 172.21.15.102:0/4288169766 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a802ace0 con 0x7fc1b00990b0 2024-09-13T17:43:53.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.288+0000 7fc1adffb640 1 -- 172.21.15.102:0/4288169766 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a802ae40 con 0x7fc1b00990b0 2024-09-13T17:43:53.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.288+0000 7fc1adffb640 1 -- 172.21.15.102:0/4288169766 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8032670 con 0x7fc1b00990b0 2024-09-13T17:43:53.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.289+0000 7fc1b551d640 1 -- 172.21.15.102:0/4288169766 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1b00990b0 msgr2=0x7fc1b00994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:53.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.289+0000 7fc1b551d640 1 --2- 172.21.15.102:0/4288169766 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1b00990b0 0x7fc1b00994f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8009880 tx=0x7fc1a80090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.289+0000 7fc1b551d640 1 -- 172.21.15.102:0/4288169766 shutdown_connections 2024-09-13T17:43:53.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.289+0000 7fc1b551d640 1 --2- 172.21.15.102:0/4288169766 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1b00990b0 0x7fc1b00994f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.289+0000 7fc1b551d640 1 --2- 172.21.15.102:0/4288169766 >> v2:172.21.15.102:3300/0 conn(0x7fc1b00982e0 0x7fc1b009e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.289+0000 7fc1b551d640 1 -- 172.21.15.102:0/4288169766 >> 172.21.15.102:0/4288169766 conn(0x7fc1b005e620 msgr2=0x7fc1b009f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:53.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.289+0000 7fc1b551d640 1 -- 172.21.15.102:0/4288169766 shutdown_connections 2024-09-13T17:43:53.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.289+0000 7fc1b551d640 1 -- 172.21.15.102:0/4288169766 wait complete. 2024-09-13T17:43:53.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.290+0000 7fc1b551d640 1 Processor -- start 2024-09-13T17:43:53.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.290+0000 7fc1b551d640 1 -- start start 2024-09-13T17:43:53.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.290+0000 7fc1b551d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1b00982e0 0x7fc1b013c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:53.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.290+0000 7fc1b551d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc1b00990b0 0x7fc1b013cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:53.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.290+0000 7fc1b551d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1b009e5d0 0x7fc1b0119440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:53.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.290+0000 7fc1b551d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b011a920 con 0x7fc1b009e5d0 2024-09-13T17:43:53.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.290+0000 7fc1b551d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b011cab0 con 0x7fc1b00982e0 2024-09-13T17:43:53.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.290+0000 7fc1b551d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b01305e0 con 0x7fc1b00990b0 2024-09-13T17:43:53.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.290+0000 7fc1aeffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1b00982e0 0x7fc1b013c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:53.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.290+0000 7fc1af7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1b009e5d0 0x7fc1b0119440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:53.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.290+0000 7fc1aeffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1b00982e0 0x7fc1b013c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:38614/0 (socket says 172.21.15.102:38614) 2024-09-13T17:43:53.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.290+0000 7fc1aeffd640 1 -- 172.21.15.102:0/843963839 learned_addr learned my addr 172.21.15.102:0/843963839 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:53.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.290+0000 7fc1ae7fc640 1 --2- 172.21.15.102:0/843963839 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc1b00990b0 0x7fc1b013cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:53.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.290+0000 7fc1aeffd640 1 -- 172.21.15.102:0/843963839 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc1b00990b0 msgr2=0x7fc1b013cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:53.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.290+0000 7fc1aeffd640 1 --2- 172.21.15.102:0/843963839 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc1b00990b0 0x7fc1b013cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.290+0000 7fc1aeffd640 1 -- 172.21.15.102:0/843963839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1b009e5d0 msgr2=0x7fc1b0119440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:53.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.290+0000 7fc1aeffd640 1 --2- 172.21.15.102:0/843963839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1b009e5d0 0x7fc1b0119440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.290+0000 7fc1aeffd640 1 -- 172.21.15.102:0/843963839 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a0009a20 con 0x7fc1b00982e0 2024-09-13T17:43:53.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.290+0000 7fc1af7fe640 1 --2- 172.21.15.102:0/843963839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1b009e5d0 0x7fc1b0119440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:53.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.290+0000 7fc1ae7fc640 1 --2- 172.21.15.102:0/843963839 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc1b00990b0 0x7fc1b013cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:53.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.291+0000 7fc1aeffd640 1 --2- 172.21.15.102:0/843963839 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1b00982e0 0x7fc1b013c8a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000c170 tx=0x7fc1a000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:53.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.291+0000 7fc1a6ffd640 1 -- 172.21.15.102:0/843963839 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0014480 con 0x7fc1b00982e0 2024-09-13T17:43:53.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.291+0000 7fc1a6ffd640 1 -- 172.21.15.102:0/843963839 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a000f040 con 0x7fc1b00982e0 2024-09-13T17:43:53.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.291+0000 7fc1b551d640 1 -- 172.21.15.102:0/843963839 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1a8009390 con 0x7fc1b00982e0 2024-09-13T17:43:53.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.291+0000 7fc1a6ffd640 1 -- 172.21.15.102:0/843963839 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0014810 con 0x7fc1b00982e0 2024-09-13T17:43:53.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.291+0000 7fc1b551d640 1 -- 172.21.15.102:0/843963839 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b011b600 con 0x7fc1b00982e0 2024-09-13T17:43:53.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.291+0000 7fc1a6ffd640 1 -- 172.21.15.102:0/843963839 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fc1a0014b80 con 0x7fc1b00982e0 2024-09-13T17:43:53.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.292+0000 7fc1a6ffd640 1 -- 172.21.15.102:0/843963839 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7fc1a0042f40 con 0x7fc1b00982e0 2024-09-13T17:43:53.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.292+0000 7fc1b551d640 1 -- 172.21.15.102:0/843963839 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b011cbf0 con 0x7fc1b00982e0 2024-09-13T17:43:53.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.296+0000 7fc1a6ffd640 1 -- 172.21.15.102:0/843963839 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fc1b011cbf0 con 0x7fc1b00982e0 2024-09-13T17:43:53.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.447+0000 7fc1b551d640 1 -- 172.21.15.102:0/843963839 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc1b011cbf0 con 0x7fc1b00982e0 2024-09-13T17:43:53.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.447+0000 7fc1a6ffd640 1 -- 172.21.15.102:0/843963839 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc1b011cbf0 con 0x7fc1b00982e0 2024-09-13T17:43:53.449 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-09-13T17:43:53.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.449+0000 7fc1b551d640 1 -- 172.21.15.102:0/843963839 >> v1:172.21.15.102:6816/34937 conn(0x7fc190031190 legacy=0x7fc190033640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:53.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.449+0000 7fc1b551d640 1 -- 172.21.15.102:0/843963839 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1b00982e0 msgr2=0x7fc1b013c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:53.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.449+0000 7fc1b551d640 1 --2- 172.21.15.102:0/843963839 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1b00982e0 0x7fc1b013c8a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000c170 tx=0x7fc1a000b040 comp rx=0 tx=0).stop 2024-09-13T17:43:53.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.449+0000 7fc1b551d640 1 -- 172.21.15.102:0/843963839 shutdown_connections 2024-09-13T17:43:53.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.449+0000 7fc1b551d640 1 --2- 172.21.15.102:0/843963839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1b009e5d0 0x7fc1b0119440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.449+0000 7fc1b551d640 1 --2- 172.21.15.102:0/843963839 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc1b00990b0 0x7fc1b013cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.449+0000 7fc1b551d640 1 --2- 172.21.15.102:0/843963839 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc1b00982e0 0x7fc1b013c8a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.449+0000 7fc1b551d640 1 -- 172.21.15.102:0/843963839 >> 172.21.15.102:0/843963839 conn(0x7fc1b005e620 msgr2=0x7fc1b0067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:53.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.449+0000 7fc1b551d640 1 -- 172.21.15.102:0/843963839 shutdown_connections 2024-09-13T17:43:53.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.449+0000 7fc1b551d640 1 -- 172.21.15.102:0/843963839 wait complete. 2024-09-13T17:43:53.462 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-13T17:43:53.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.557+0000 7f49d5b9d640 1 Processor -- start 2024-09-13T17:43:53.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.557+0000 7f49d5b9d640 1 -- start start 2024-09-13T17:43:53.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.557+0000 7f49d5b9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f49d0098a10 0x7f49d0098330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:53.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.557+0000 7f49d5b9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f49d009ae60 0x7f49d012c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:53.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.557+0000 7f49d5b9d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f49d009b250 0x7f49d012c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:53.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.557+0000 7f49d5b9d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49d00975e0 con 0x7f49d009b250 2024-09-13T17:43:53.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.557+0000 7f49d5b9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d009bca0 con 0x7f49d0098a10 2024-09-13T17:43:53.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.557+0000 7f49d5b9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f49d0099540 con 0x7f49d009ae60 2024-09-13T17:43:53.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.558+0000 7f49ceffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f49d009ae60 0x7f49d012c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:53.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.558+0000 7f49cffff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f49d009b250 0x7f49d012c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:53.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.558+0000 7f49cf7fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f49d0098a10 0x7f49d0098330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:53.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.558+0000 7f49cf7fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f49d0098a10 0x7f49d0098330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:38624/0 (socket says 172.21.15.102:38624) 2024-09-13T17:43:53.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.558+0000 7f49cf7fe640 1 -- 172.21.15.102:0/697854252 learned_addr learned my addr 172.21.15.102:0/697854252 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:53.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.558+0000 7f49cffff640 1 --2- 172.21.15.102:0/697854252 >> v2:172.21.15.102:3300/0 conn(0x7f49d009b250 0x7f49d012c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:46262/0 (socket says 172.21.15.102:46262) 2024-09-13T17:43:53.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.558+0000 7f49ceffd640 1 -- 172.21.15.102:0/697854252 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f49d0098a10 msgr2=0x7f49d0098330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:53.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.558+0000 7f49ceffd640 1 --2- 172.21.15.102:0/697854252 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f49d0098a10 0x7f49d0098330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.558+0000 7f49ceffd640 1 -- 172.21.15.102:0/697854252 >> v2:172.21.15.102:3300/0 conn(0x7f49d009b250 msgr2=0x7f49d012c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:53.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.558+0000 7f49ceffd640 1 --2- 172.21.15.102:0/697854252 >> v2:172.21.15.102:3300/0 conn(0x7f49d009b250 0x7f49d012c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.558+0000 7f49ceffd640 1 -- 172.21.15.102:0/697854252 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d0097890 con 0x7f49d009ae60 2024-09-13T17:43:53.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.558+0000 7f49cf7fe640 1 --2- 172.21.15.102:0/697854252 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f49d0098a10 0x7f49d0098330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:53.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.558+0000 7f49cffff640 1 --2- 172.21.15.102:0/697854252 >> v2:172.21.15.102:3300/0 conn(0x7f49d009b250 0x7f49d012c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:53.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.559+0000 7f49ceffd640 1 --2- 172.21.15.102:0/697854252 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f49d009ae60 0x7f49d012c160 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f49c0009850 tx=0x7f49c00090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=355e1e7e16d00476 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:53.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.559+0000 7f49ce7fc640 1 -- 172.21.15.102:0/697854252 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c002abc0 con 0x7f49d009ae60 2024-09-13T17:43:53.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.559+0000 7f49ce7fc640 1 -- 172.21.15.102:0/697854252 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c002ad20 con 0x7f49d009ae60 2024-09-13T17:43:53.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.559+0000 7f49ce7fc640 1 -- 172.21.15.102:0/697854252 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c0032670 con 0x7f49d009ae60 2024-09-13T17:43:53.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.559+0000 7f49d5b9d640 1 -- 172.21.15.102:0/697854252 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f49d009ae60 msgr2=0x7f49d012c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:53.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.559+0000 7f49d5b9d640 1 --2- 172.21.15.102:0/697854252 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f49d009ae60 0x7f49d012c160 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f49c0009850 tx=0x7f49c00090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.559+0000 7f49d5b9d640 1 -- 172.21.15.102:0/697854252 shutdown_connections 2024-09-13T17:43:53.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.559+0000 7f49d5b9d640 1 --2- 172.21.15.102:0/697854252 >> v2:172.21.15.102:3300/0 conn(0x7f49d009b250 0x7f49d012c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.559+0000 7f49d5b9d640 1 --2- 172.21.15.102:0/697854252 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f49d009ae60 0x7f49d012c160 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.559+0000 7f49d5b9d640 1 --2- 172.21.15.102:0/697854252 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f49d0098a10 0x7f49d0098330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.559+0000 7f49d5b9d640 1 -- 172.21.15.102:0/697854252 >> 172.21.15.102:0/697854252 conn(0x7f49d005e610 msgr2=0x7f49d009f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:53.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.559+0000 7f49d5b9d640 1 -- 172.21.15.102:0/697854252 shutdown_connections 2024-09-13T17:43:53.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.559+0000 7f49d5b9d640 1 -- 172.21.15.102:0/697854252 wait complete. 2024-09-13T17:43:53.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.560+0000 7f49d5b9d640 1 Processor -- start 2024-09-13T17:43:53.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.560+0000 7f49d5b9d640 1 -- start start 2024-09-13T17:43:53.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.561+0000 7f49d5b9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f49d0098a10 0x7f49d01298c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:53.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.561+0000 7f49d5b9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49d009ae60 0x7f49d0119960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:53.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.561+0000 7f49d5b9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f49d009b250 0x7f49d0119ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:53.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.561+0000 7f49d5b9d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d011b9d0 con 0x7f49d009ae60 2024-09-13T17:43:53.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.561+0000 7f49d5b9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d011b490 con 0x7f49d0098a10 2024-09-13T17:43:53.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.561+0000 7f49d5b9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f49d0129e00 con 0x7f49d009b250 2024-09-13T17:43:53.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.561+0000 7f49ceffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49d009ae60 0x7f49d0119960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:53.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.561+0000 7f49cffff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f49d009b250 0x7f49d0119ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:53.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.561+0000 7f49cf7fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f49d0098a10 0x7f49d01298c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:53.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.561+0000 7f49ceffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49d009ae60 0x7f49d0119960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:46266/0 (socket says 172.21.15.102:46266) 2024-09-13T17:43:53.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.561+0000 7f49ceffd640 1 -- 172.21.15.102:0/2516912234 learned_addr learned my addr 172.21.15.102:0/2516912234 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:53.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.561+0000 7f49ceffd640 1 -- 172.21.15.102:0/2516912234 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f49d009b250 msgr2=0x7f49d0119ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:53.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.561+0000 7f49ceffd640 1 --2- 172.21.15.102:0/2516912234 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f49d009b250 0x7f49d0119ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.561+0000 7f49ceffd640 1 -- 172.21.15.102:0/2516912234 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f49d0098a10 msgr2=0x7f49d01298c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:53.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.561+0000 7f49ceffd640 1 --2- 172.21.15.102:0/2516912234 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f49d0098a10 0x7f49d01298c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.561+0000 7f49ceffd640 1 -- 172.21.15.102:0/2516912234 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b8009450 con 0x7f49d009ae60 2024-09-13T17:43:53.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.561+0000 7f49cf7fe640 1 --2- 172.21.15.102:0/2516912234 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f49d0098a10 0x7f49d01298c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:53.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.561+0000 7f49ceffd640 1 --2- 172.21.15.102:0/2516912234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49d009ae60 0x7f49d0119960 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f49c002a4f0 tx=0x7f49c0032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:53.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.561+0000 7f49c77fe640 1 -- 172.21.15.102:0/2516912234 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c003d720 con 0x7f49d009ae60 2024-09-13T17:43:53.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.561+0000 7f49c77fe640 1 -- 172.21.15.102:0/2516912234 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c002ae90 con 0x7f49d009ae60 2024-09-13T17:43:53.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.561+0000 7f49c77fe640 1 -- 172.21.15.102:0/2516912234 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c003dab0 con 0x7f49d009ae60 2024-09-13T17:43:53.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.561+0000 7f49d5b9d640 1 -- 172.21.15.102:0/2516912234 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c8009a20 con 0x7f49d009ae60 2024-09-13T17:43:53.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.561+0000 7f49d5b9d640 1 -- 172.21.15.102:0/2516912234 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c0009390 con 0x7f49d009ae60 2024-09-13T17:43:53.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.562+0000 7f49c77fe640 1 -- 172.21.15.102:0/2516912234 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f49c0039040 con 0x7f49d009ae60 2024-09-13T17:43:53.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.563+0000 7f49c77fe640 1 -- 172.21.15.102:0/2516912234 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f49c0061640 con 0x7f49d009ae60 2024-09-13T17:43:53.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.563+0000 7f49d5b9d640 1 -- 172.21.15.102:0/2516912234 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49d011cd40 con 0x7f49d009ae60 2024-09-13T17:43:53.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.567+0000 7f49c77fe640 1 -- 172.21.15.102:0/2516912234 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f49c0037020 con 0x7f49d009ae60 2024-09-13T17:43:53.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.717+0000 7f49d5b9d640 1 -- 172.21.15.102:0/2516912234 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f49d0064600 con 0x7f49d009ae60 2024-09-13T17:43:53.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.717+0000 7f49c77fe640 1 -- 172.21.15.102:0/2516912234 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f49c0042070 con 0x7f49d009ae60 2024-09-13T17:43:53.719 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-09-13T17:43:53.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.719+0000 7f49d5b9d640 1 -- 172.21.15.102:0/2516912234 >> v1:172.21.15.102:6816/34937 conn(0x7f49b0031160 legacy=0x7f49b0033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:53.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.719+0000 7f49d5b9d640 1 -- 172.21.15.102:0/2516912234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49d009ae60 msgr2=0x7f49d0119960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:53.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.719+0000 7f49d5b9d640 1 --2- 172.21.15.102:0/2516912234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49d009ae60 0x7f49d0119960 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f49c002a4f0 tx=0x7f49c0032000 comp rx=0 tx=0).stop 2024-09-13T17:43:53.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.719+0000 7f49d5b9d640 1 -- 172.21.15.102:0/2516912234 shutdown_connections 2024-09-13T17:43:53.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.719+0000 7f49d5b9d640 1 --2- 172.21.15.102:0/2516912234 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f49d009b250 0x7f49d0119ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.719+0000 7f49d5b9d640 1 --2- 172.21.15.102:0/2516912234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49d009ae60 0x7f49d0119960 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.719+0000 7f49d5b9d640 1 --2- 172.21.15.102:0/2516912234 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f49d0098a10 0x7f49d01298c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.719+0000 7f49d5b9d640 1 -- 172.21.15.102:0/2516912234 >> 172.21.15.102:0/2516912234 conn(0x7f49d005e610 msgr2=0x7f49d012c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:53.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.719+0000 7f49d5b9d640 1 -- 172.21.15.102:0/2516912234 shutdown_connections 2024-09-13T17:43:53.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.719+0000 7f49d5b9d640 1 -- 172.21.15.102:0/2516912234 wait complete. 2024-09-13T17:43:53.731 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:43:53.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.825+0000 7f273971d640 1 Processor -- start 2024-09-13T17:43:53.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.825+0000 7f273971d640 1 -- start start 2024-09-13T17:43:53.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.826+0000 7f273971d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f273409c350 0x7f273406e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:53.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.826+0000 7f273971d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f273406e120 0x7f273412da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:53.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.826+0000 7f273971d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f273406d150 con 0x7f273406eba0 2024-09-13T17:43:53.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.826+0000 7f273971d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f273409c720 con 0x7f273406e120 2024-09-13T17:43:53.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.826+0000 7f273971d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f273409cc90 con 0x7f273409c350 2024-09-13T17:43:53.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.826+0000 7f27337fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f273406e120 0x7f273412da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:53.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.826+0000 7f2732ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f273409c350 0x7f273406e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:53.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.826+0000 7f27327fc640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f273406eba0 0x7f273412d2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:58890/0 (socket says 172.21.15.102:58890) 2024-09-13T17:43:53.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.826+0000 7f27337fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f273406e120 0x7f273412da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:46274/0 (socket says 172.21.15.102:46274) 2024-09-13T17:43:53.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.826+0000 7f27327fc640 1 -- 172.21.15.102:0/3491893041 learned_addr learned my addr 172.21.15.102:0/3491893041 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:53.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.826+0000 7f2731ffb640 1 -- 172.21.15.102:0/3491893041 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4085931130 0 0) 0x7f273406d150 con 0x7f273406eba0 2024-09-13T17:43:53.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.826+0000 7f2731ffb640 1 -- 172.21.15.102:0/3491893041 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f271c0034e0 con 0x7f273406eba0 2024-09-13T17:43:53.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.826+0000 7f27337fe640 1 -- 172.21.15.102:0/3491893041 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f273409c350 msgr2=0x7f273406e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:53.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.826+0000 7f27337fe640 1 --2- 172.21.15.102:0/3491893041 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f273409c350 0x7f273406e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.826+0000 7f27337fe640 1 -- 172.21.15.102:0/3491893041 >> v1:172.21.15.102:6789/0 conn(0x7f273406eba0 legacy=0x7f273412d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:53.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.826+0000 7f27337fe640 1 -- 172.21.15.102:0/3491893041 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f273409d010 con 0x7f273406e120 2024-09-13T17:43:53.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.826+0000 7f2732ffd640 1 --2- 172.21.15.102:0/3491893041 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f273409c350 0x7f273406e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:53.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.827+0000 7f27337fe640 1 --2- 172.21.15.102:0/3491893041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f273406e120 0x7f273412da00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f272c009850 tx=0x7f272c0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b9b1818d8454ef server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:53.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.827+0000 7f2731ffb640 1 -- 172.21.15.102:0/3491893041 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272c02ad00 con 0x7f273406e120 2024-09-13T17:43:53.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.827+0000 7f2731ffb640 1 -- 172.21.15.102:0/3491893041 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f272c02ae60 con 0x7f273406e120 2024-09-13T17:43:53.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.827+0000 7f2731ffb640 1 -- 172.21.15.102:0/3491893041 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272c032670 con 0x7f273406e120 2024-09-13T17:43:53.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.827+0000 7f273971d640 1 -- 172.21.15.102:0/3491893041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f273406e120 msgr2=0x7f273412da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:53.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.827+0000 7f273971d640 1 --2- 172.21.15.102:0/3491893041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f273406e120 0x7f273412da00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f272c009850 tx=0x7f272c0090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.827+0000 7f273971d640 1 -- 172.21.15.102:0/3491893041 shutdown_connections 2024-09-13T17:43:53.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.827+0000 7f273971d640 1 --2- 172.21.15.102:0/3491893041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f273406e120 0x7f273412da00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.828+0000 7f273971d640 1 --2- 172.21.15.102:0/3491893041 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f273409c350 0x7f273406e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.828+0000 7f273971d640 1 -- 172.21.15.102:0/3491893041 >> 172.21.15.102:0/3491893041 conn(0x7f27340a0950 msgr2=0x7f27340a0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:53.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.828+0000 7f273971d640 1 -- 172.21.15.102:0/3491893041 shutdown_connections 2024-09-13T17:43:53.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.828+0000 7f273971d640 1 -- 172.21.15.102:0/3491893041 wait complete. 2024-09-13T17:43:53.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.828+0000 7f273971d640 1 Processor -- start 2024-09-13T17:43:53.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.828+0000 7f273971d640 1 -- start start 2024-09-13T17:43:53.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.829+0000 7f273971d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f273406e120 0x7f273413eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:53.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.829+0000 7f273971d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f273406eba0 0x7f273413f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:53.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.829+0000 7f273971d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f273409c350 0x7f2734068240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:53.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.829+0000 7f273971d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273406c170 con 0x7f273406e120 2024-09-13T17:43:53.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.829+0000 7f273971d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f273411c180 con 0x7f273409c350 2024-09-13T17:43:53.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.829+0000 7f273971d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f273413c990 con 0x7f273406eba0 2024-09-13T17:43:53.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.829+0000 7f2732ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f273406e120 0x7f273413eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:53.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.829+0000 7f27327fc640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f273406eba0 0x7f273413f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:53.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.829+0000 7f27337fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f273409c350 0x7f2734068240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:53.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.829+0000 7f2732ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f273406e120 0x7f273413eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:46284/0 (socket says 172.21.15.102:46284) 2024-09-13T17:43:53.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.829+0000 7f2732ffd640 1 -- 172.21.15.102:0/2107367667 learned_addr learned my addr 172.21.15.102:0/2107367667 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:53.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.829+0000 7f2732ffd640 1 -- 172.21.15.102:0/2107367667 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f273406eba0 msgr2=0x7f273413f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:53.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.829+0000 7f2732ffd640 1 --2- 172.21.15.102:0/2107367667 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f273406eba0 0x7f273413f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.829+0000 7f2732ffd640 1 -- 172.21.15.102:0/2107367667 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f273409c350 msgr2=0x7f2734068240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:53.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.829+0000 7f2732ffd640 1 --2- 172.21.15.102:0/2107367667 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f273409c350 0x7f2734068240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.829+0000 7f2732ffd640 1 -- 172.21.15.102:0/2107367667 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2724009a20 con 0x7f273406e120 2024-09-13T17:43:53.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.829+0000 7f27327fc640 1 --2- 172.21.15.102:0/2107367667 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f273406eba0 0x7f273413f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:53.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.829+0000 7f2732ffd640 1 --2- 172.21.15.102:0/2107367667 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f273406e120 0x7f273413eb90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f272400c170 tx=0x7f272400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:53.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.830+0000 7f271affd640 1 -- 172.21.15.102:0/2107367667 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2724014480 con 0x7f273406e120 2024-09-13T17:43:53.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.830+0000 7f271affd640 1 -- 172.21.15.102:0/2107367667 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f272400f040 con 0x7f273406e120 2024-09-13T17:43:53.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.830+0000 7f273971d640 1 -- 172.21.15.102:0/2107367667 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f272c009390 con 0x7f273406e120 2024-09-13T17:43:53.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.830+0000 7f271affd640 1 -- 172.21.15.102:0/2107367667 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27240147f0 con 0x7f273406e120 2024-09-13T17:43:53.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.830+0000 7f273971d640 1 -- 172.21.15.102:0/2107367667 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f273409a300 con 0x7f273406e120 2024-09-13T17:43:53.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.830+0000 7f271affd640 1 -- 172.21.15.102:0/2107367667 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f2724014b60 con 0x7f273406e120 2024-09-13T17:43:53.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.831+0000 7f271affd640 1 -- 172.21.15.102:0/2107367667 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f27240430a0 con 0x7f273406e120 2024-09-13T17:43:53.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.831+0000 7f273971d640 1 -- 172.21.15.102:0/2107367667 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2734064600 con 0x7f273406e120 2024-09-13T17:43:53.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.835+0000 7f271affd640 1 -- 172.21.15.102:0/2107367667 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f2724012570 con 0x7f273406e120 2024-09-13T17:43:53.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.984+0000 7f273971d640 1 -- 172.21.15.102:0/2107367667 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f273409af10 con 0x7f273406e120 2024-09-13T17:43:53.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.985+0000 7f271affd640 1 -- 172.21.15.102:0/2107367667 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2724007b30 con 0x7f273406e120 2024-09-13T17:43:53.987 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-09-13T17:43:53.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.986+0000 7f273971d640 1 -- 172.21.15.102:0/2107367667 >> v1:172.21.15.102:6816/34937 conn(0x7f2720031160 legacy=0x7f2720033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:53.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.986+0000 7f273971d640 1 -- 172.21.15.102:0/2107367667 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f273406e120 msgr2=0x7f273413eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:53.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.986+0000 7f273971d640 1 --2- 172.21.15.102:0/2107367667 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f273406e120 0x7f273413eb90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f272400c170 tx=0x7f272400b040 comp rx=0 tx=0).stop 2024-09-13T17:43:53.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.986+0000 7f273971d640 1 -- 172.21.15.102:0/2107367667 shutdown_connections 2024-09-13T17:43:53.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.986+0000 7f273971d640 1 --2- 172.21.15.102:0/2107367667 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f273409c350 0x7f2734068240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.986+0000 7f273971d640 1 --2- 172.21.15.102:0/2107367667 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f273406eba0 0x7f273413f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.986+0000 7f273971d640 1 --2- 172.21.15.102:0/2107367667 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f273406e120 0x7f273413eb90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.986+0000 7f273971d640 1 -- 172.21.15.102:0/2107367667 >> 172.21.15.102:0/2107367667 conn(0x7f27340a0950 msgr2=0x7f273412e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:53.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.986+0000 7f273971d640 1 -- 172.21.15.102:0/2107367667 shutdown_connections 2024-09-13T17:43:53.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:53.987+0000 7f273971d640 1 -- 172.21.15.102:0/2107367667 wait complete. 2024-09-13T17:43:53.999 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:43:54.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.093+0000 7fc839e34640 1 Processor -- start 2024-09-13T17:43:54.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.093+0000 7fc839e34640 1 -- start start 2024-09-13T17:43:54.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.094+0000 7fc839e34640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc834097d30 0x7fc834132610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:54.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.094+0000 7fc839e34640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc834132b50 0x7fc834134f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:54.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.094+0000 7fc839e34640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc83406a850 con 0x7fc834096df0 2024-09-13T17:43:54.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.094+0000 7fc839e34640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc834098290 con 0x7fc834097d30 2024-09-13T17:43:54.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.094+0000 7fc839e34640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc834096bf0 con 0x7fc834132b50 2024-09-13T17:43:54.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.094+0000 7fc832ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc834097d30 0x7fc834132610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:54.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.094+0000 7fc8337fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fc834096df0 0x7fc8340971c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:58892/0 (socket says 172.21.15.102:58892) 2024-09-13T17:43:54.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.094+0000 7fc832ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc834097d30 0x7fc834132610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:38660/0 (socket says 172.21.15.102:38660) 2024-09-13T17:43:54.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.094+0000 7fc8337fe640 1 -- 172.21.15.102:0/130130708 learned_addr learned my addr 172.21.15.102:0/130130708 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:54.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.094+0000 7fc833fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc834132b50 0x7fc834134f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:54.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.094+0000 7fc832ffd640 1 -- 172.21.15.102:0/130130708 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc834132b50 msgr2=0x7fc834134f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:54.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.094+0000 7fc832ffd640 1 --2- 172.21.15.102:0/130130708 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc834132b50 0x7fc834134f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.094+0000 7fc8327fc640 1 -- 172.21.15.102:0/130130708 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1660959511 0 0) 0x7fc83406a850 con 0x7fc834096df0 2024-09-13T17:43:54.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.094+0000 7fc832ffd640 1 -- 172.21.15.102:0/130130708 >> v1:172.21.15.102:6789/0 conn(0x7fc834096df0 legacy=0x7fc8340971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:54.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.094+0000 7fc832ffd640 1 -- 172.21.15.102:0/130130708 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc83406a590 con 0x7fc834097d30 2024-09-13T17:43:54.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.094+0000 7fc833fff640 1 --2- 172.21.15.102:0/130130708 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc834132b50 0x7fc834134f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:54.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.095+0000 7fc832ffd640 1 --2- 172.21.15.102:0/130130708 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc834097d30 0x7fc834132610 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc828009880 tx=0x7fc8280090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e2826a5c3627e92 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:54.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.095+0000 7fc8327fc640 1 -- 172.21.15.102:0/130130708 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82802acd0 con 0x7fc834097d30 2024-09-13T17:43:54.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.095+0000 7fc8327fc640 1 -- 172.21.15.102:0/130130708 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc82802ae30 con 0x7fc834097d30 2024-09-13T17:43:54.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.095+0000 7fc8327fc640 1 -- 172.21.15.102:0/130130708 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc828032670 con 0x7fc834097d30 2024-09-13T17:43:54.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.095+0000 7fc839e34640 1 -- 172.21.15.102:0/130130708 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc834097d30 msgr2=0x7fc834132610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:54.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.095+0000 7fc839e34640 1 --2- 172.21.15.102:0/130130708 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc834097d30 0x7fc834132610 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc828009880 tx=0x7fc8280090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.096+0000 7fc839e34640 1 -- 172.21.15.102:0/130130708 shutdown_connections 2024-09-13T17:43:54.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.096+0000 7fc839e34640 1 --2- 172.21.15.102:0/130130708 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc834132b50 0x7fc834134f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.096+0000 7fc839e34640 1 --2- 172.21.15.102:0/130130708 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc834097d30 0x7fc834132610 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.096+0000 7fc839e34640 1 -- 172.21.15.102:0/130130708 >> 172.21.15.102:0/130130708 conn(0x7fc83409e720 msgr2=0x7fc83409eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:54.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.096+0000 7fc839e34640 1 -- 172.21.15.102:0/130130708 shutdown_connections 2024-09-13T17:43:54.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.096+0000 7fc839e34640 1 -- 172.21.15.102:0/130130708 wait complete. 2024-09-13T17:43:54.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.097+0000 7fc839e34640 1 Processor -- start 2024-09-13T17:43:54.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.097+0000 7fc839e34640 1 -- start start 2024-09-13T17:43:54.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.097+0000 7fc839e34640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc834096df0 0x7fc834117f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:54.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.097+0000 7fc839e34640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc834097d30 0x7fc834118470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:54.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.097+0000 7fc839e34640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc834132b50 0x7fc8341189b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:54.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.097+0000 7fc839e34640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83411b990 con 0x7fc834096df0 2024-09-13T17:43:54.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.097+0000 7fc839e34640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc834069df0 con 0x7fc834097d30 2024-09-13T17:43:54.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.097+0000 7fc839e34640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc8341193a0 con 0x7fc834132b50 2024-09-13T17:43:54.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.097+0000 7fc8337fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc834096df0 0x7fc834117f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:54.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.097+0000 7fc832ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc834097d30 0x7fc834118470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:54.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.097+0000 7fc833fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc834132b50 0x7fc8341189b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:54.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.097+0000 7fc832ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc834097d30 0x7fc834118470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:38672/0 (socket says 172.21.15.102:38672) 2024-09-13T17:43:54.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.097+0000 7fc8337fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc834096df0 0x7fc834117f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:46288/0 (socket says 172.21.15.102:46288) 2024-09-13T17:43:54.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.097+0000 7fc832ffd640 1 -- 172.21.15.102:0/4224816053 learned_addr learned my addr 172.21.15.102:0/4224816053 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:54.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.097+0000 7fc8337fe640 1 -- 172.21.15.102:0/4224816053 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc834132b50 msgr2=0x7fc8341189b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:54.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.097+0000 7fc8337fe640 1 --2- 172.21.15.102:0/4224816053 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc834132b50 0x7fc8341189b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.097+0000 7fc8337fe640 1 -- 172.21.15.102:0/4224816053 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc834097d30 msgr2=0x7fc834118470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:54.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.097+0000 7fc8337fe640 1 --2- 172.21.15.102:0/4224816053 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc834097d30 0x7fc834118470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.097+0000 7fc8337fe640 1 -- 172.21.15.102:0/4224816053 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc82c009a20 con 0x7fc834096df0 2024-09-13T17:43:54.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.098+0000 7fc832ffd640 1 --2- 172.21.15.102:0/4224816053 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc834097d30 0x7fc834118470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:54.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.098+0000 7fc833fff640 1 --2- 172.21.15.102:0/4224816053 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc834132b50 0x7fc8341189b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:54.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.098+0000 7fc8337fe640 1 --2- 172.21.15.102:0/4224816053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc834096df0 0x7fc834117f30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc824000c00 tx=0x7fc824007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:54.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.098+0000 7fc81b7fe640 1 -- 172.21.15.102:0/4224816053 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc824005aa0 con 0x7fc834096df0 2024-09-13T17:43:54.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.098+0000 7fc81b7fe640 1 -- 172.21.15.102:0/4224816053 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc824005c40 con 0x7fc834096df0 2024-09-13T17:43:54.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.098+0000 7fc839e34640 1 -- 172.21.15.102:0/4224816053 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc828009390 con 0x7fc834096df0 2024-09-13T17:43:54.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.098+0000 7fc81b7fe640 1 -- 172.21.15.102:0/4224816053 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8240070d0 con 0x7fc834096df0 2024-09-13T17:43:54.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.098+0000 7fc839e34640 1 -- 172.21.15.102:0/4224816053 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc83411bf50 con 0x7fc834096df0 2024-09-13T17:43:54.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.099+0000 7fc81b7fe640 1 -- 172.21.15.102:0/4224816053 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fc824007500 con 0x7fc834096df0 2024-09-13T17:43:54.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.099+0000 7fc81b7fe640 1 -- 172.21.15.102:0/4224816053 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7fc8240414d0 con 0x7fc834096df0 2024-09-13T17:43:54.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.099+0000 7fc839e34640 1 -- 172.21.15.102:0/4224816053 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc83411b6b0 con 0x7fc834096df0 2024-09-13T17:43:54.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.103+0000 7fc81b7fe640 1 -- 172.21.15.102:0/4224816053 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fc824017030 con 0x7fc834096df0 2024-09-13T17:43:54.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.255+0000 7fc839e34640 1 -- 172.21.15.102:0/4224816053 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc834064000 con 0x7fc834096df0 2024-09-13T17:43:54.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.255+0000 7fc81b7fe640 1 -- 172.21.15.102:0/4224816053 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc824017070 con 0x7fc834096df0 2024-09-13T17:43:54.257 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-09-13T17:43:54.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.257+0000 7fc839e34640 1 -- 172.21.15.102:0/4224816053 >> v1:172.21.15.102:6816/34937 conn(0x7fc820031160 legacy=0x7fc820033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:54.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.257+0000 7fc839e34640 1 -- 172.21.15.102:0/4224816053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc834096df0 msgr2=0x7fc834117f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:54.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.257+0000 7fc839e34640 1 --2- 172.21.15.102:0/4224816053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc834096df0 0x7fc834117f30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc824000c00 tx=0x7fc824007c80 comp rx=0 tx=0).stop 2024-09-13T17:43:54.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.257+0000 7fc839e34640 1 -- 172.21.15.102:0/4224816053 shutdown_connections 2024-09-13T17:43:54.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.257+0000 7fc839e34640 1 --2- 172.21.15.102:0/4224816053 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc834132b50 0x7fc8341189b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.257+0000 7fc839e34640 1 --2- 172.21.15.102:0/4224816053 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc834097d30 0x7fc834118470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.257+0000 7fc839e34640 1 --2- 172.21.15.102:0/4224816053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc834096df0 0x7fc834117f30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.257+0000 7fc839e34640 1 -- 172.21.15.102:0/4224816053 >> 172.21.15.102:0/4224816053 conn(0x7fc83409e720 msgr2=0x7fc83409d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:54.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.257+0000 7fc839e34640 1 -- 172.21.15.102:0/4224816053 shutdown_connections 2024-09-13T17:43:54.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.257+0000 7fc839e34640 1 -- 172.21.15.102:0/4224816053 wait complete. 2024-09-13T17:43:54.270 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:43:54.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.362+0000 7efd9599d640 1 Processor -- start 2024-09-13T17:43:54.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.362+0000 7efd9599d640 1 -- start start 2024-09-13T17:43:54.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.363+0000 7efd9599d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd90068920 0x7efd9009d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:54.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.363+0000 7efd9599d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efd9009e020 0x7efd9012fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:54.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.363+0000 7efd9599d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd9009e470 con 0x7efd9009dc50 2024-09-13T17:43:54.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.363+0000 7efd9599d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd90068cf0 con 0x7efd90068920 2024-09-13T17:43:54.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.363+0000 7efd9599d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efd900695a0 con 0x7efd9009e020 2024-09-13T17:43:54.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.363+0000 7efd8effd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd90068920 0x7efd9009d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:54.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.363+0000 7efd8e7fc640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7efd9009dc50 0x7efd9012f310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:58906/0 (socket says 172.21.15.102:58906) 2024-09-13T17:43:54.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.363+0000 7efd8e7fc640 1 -- 172.21.15.102:0/1474755556 learned_addr learned my addr 172.21.15.102:0/1474755556 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:54.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.363+0000 7efd8f7fe640 1 --2- 172.21.15.102:0/1474755556 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efd9009e020 0x7efd9012fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:54.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.363+0000 7efd8dffb640 1 -- 172.21.15.102:0/1474755556 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3395373439 0 0) 0x7efd9009e470 con 0x7efd9009dc50 2024-09-13T17:43:54.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.363+0000 7efd8f7fe640 1 -- 172.21.15.102:0/1474755556 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd90068920 msgr2=0x7efd9009d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:54.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.363+0000 7efd8f7fe640 1 --2- 172.21.15.102:0/1474755556 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd90068920 0x7efd9009d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.363+0000 7efd8f7fe640 1 -- 172.21.15.102:0/1474755556 >> v1:172.21.15.102:6789/0 conn(0x7efd9009dc50 legacy=0x7efd9012f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:54.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.363+0000 7efd8f7fe640 1 -- 172.21.15.102:0/1474755556 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9009f090 con 0x7efd9009e020 2024-09-13T17:43:54.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.363+0000 7efd8effd640 1 --2- 172.21.15.102:0/1474755556 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd90068920 0x7efd9009d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:54.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.364+0000 7efd8f7fe640 1 --2- 172.21.15.102:0/1474755556 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efd9009e020 0x7efd9012fa20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efd88009880 tx=0x7efd880090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69206547f732d2e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:54.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.364+0000 7efd8dffb640 1 -- 172.21.15.102:0/1474755556 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8802ac50 con 0x7efd9009e020 2024-09-13T17:43:54.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.364+0000 7efd8dffb640 1 -- 172.21.15.102:0/1474755556 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8802adb0 con 0x7efd9009e020 2024-09-13T17:43:54.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.364+0000 7efd8dffb640 1 -- 172.21.15.102:0/1474755556 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd88032670 con 0x7efd9009e020 2024-09-13T17:43:54.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.364+0000 7efd9599d640 1 -- 172.21.15.102:0/1474755556 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efd9009e020 msgr2=0x7efd9012fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:54.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.364+0000 7efd9599d640 1 --2- 172.21.15.102:0/1474755556 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efd9009e020 0x7efd9012fa20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efd88009880 tx=0x7efd880090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.365+0000 7efd9599d640 1 -- 172.21.15.102:0/1474755556 shutdown_connections 2024-09-13T17:43:54.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.365+0000 7efd9599d640 1 --2- 172.21.15.102:0/1474755556 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efd9009e020 0x7efd9012fa20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.365+0000 7efd9599d640 1 --2- 172.21.15.102:0/1474755556 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd90068920 0x7efd9009d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.365+0000 7efd9599d640 1 -- 172.21.15.102:0/1474755556 >> 172.21.15.102:0/1474755556 conn(0x7efd9006f320 msgr2=0x7efd900a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:54.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.365+0000 7efd9599d640 1 -- 172.21.15.102:0/1474755556 shutdown_connections 2024-09-13T17:43:54.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.365+0000 7efd9599d640 1 -- 172.21.15.102:0/1474755556 wait complete. 2024-09-13T17:43:54.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.366+0000 7efd9599d640 1 Processor -- start 2024-09-13T17:43:54.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.366+0000 7efd9599d640 1 -- start start 2024-09-13T17:43:54.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.366+0000 7efd9599d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efd90068920 0x7efd9011e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:54.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.366+0000 7efd9599d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd9009dc50 0x7efd9011af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:54.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.366+0000 7efd9599d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd9009e020 0x7efd9011b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:54.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.366+0000 7efd9599d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd900a1b10 con 0x7efd9009dc50 2024-09-13T17:43:54.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.366+0000 7efd9599d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd9009e620 con 0x7efd9009e020 2024-09-13T17:43:54.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.366+0000 7efd9599d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efd9011acb0 con 0x7efd90068920 2024-09-13T17:43:54.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.366+0000 7efd8effd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efd90068920 0x7efd9011e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:54.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.366+0000 7efd8effd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efd90068920 0x7efd9011e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:36288/0 (socket says 172.21.15.102:36288) 2024-09-13T17:43:54.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.366+0000 7efd8effd640 1 -- 172.21.15.102:0/1586210509 learned_addr learned my addr 172.21.15.102:0/1586210509 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:54.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.366+0000 7efd8e7fc640 1 --2- 172.21.15.102:0/1586210509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd9009dc50 0x7efd9011af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:54.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.366+0000 7efd8f7fe640 1 --2- 172.21.15.102:0/1586210509 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd9009e020 0x7efd9011b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:54.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.366+0000 7efd8effd640 1 -- 172.21.15.102:0/1586210509 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd9009e020 msgr2=0x7efd9011b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:54.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.366+0000 7efd8effd640 1 --2- 172.21.15.102:0/1586210509 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd9009e020 0x7efd9011b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.366+0000 7efd8effd640 1 -- 172.21.15.102:0/1586210509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd9009dc50 msgr2=0x7efd9011af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:54.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.366+0000 7efd8effd640 1 --2- 172.21.15.102:0/1586210509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd9009dc50 0x7efd9011af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.366+0000 7efd8effd640 1 -- 172.21.15.102:0/1586210509 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd80009a20 con 0x7efd90068920 2024-09-13T17:43:54.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.366+0000 7efd8e7fc640 1 --2- 172.21.15.102:0/1586210509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd9009dc50 0x7efd9011af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:54.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.367+0000 7efd8effd640 1 --2- 172.21.15.102:0/1586210509 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efd90068920 0x7efd9011e790 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efd8000c1a0 tx=0x7efd8000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:54.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.367+0000 7efd86ffd640 1 -- 172.21.15.102:0/1586210509 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd80014480 con 0x7efd90068920 2024-09-13T17:43:54.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.367+0000 7efd86ffd640 1 -- 172.21.15.102:0/1586210509 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8000f040 con 0x7efd90068920 2024-09-13T17:43:54.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.367+0000 7efd86ffd640 1 -- 172.21.15.102:0/1586210509 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd80014810 con 0x7efd90068920 2024-09-13T17:43:54.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.367+0000 7efd9599d640 1 -- 172.21.15.102:0/1586210509 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd88009390 con 0x7efd90068920 2024-09-13T17:43:54.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.367+0000 7efd9599d640 1 -- 172.21.15.102:0/1586210509 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd9011ba10 con 0x7efd90068920 2024-09-13T17:43:54.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.367+0000 7efd86ffd640 1 -- 172.21.15.102:0/1586210509 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7efd80014b80 con 0x7efd90068920 2024-09-13T17:43:54.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.368+0000 7efd86ffd640 1 -- 172.21.15.102:0/1586210509 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7efd80042f80 con 0x7efd90068920 2024-09-13T17:43:54.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.368+0000 7efd9599d640 1 -- 172.21.15.102:0/1586210509 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd90064000 con 0x7efd90068920 2024-09-13T17:43:54.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.372+0000 7efd86ffd640 1 -- 172.21.15.102:0/1586210509 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7efd80014e30 con 0x7efd90068920 2024-09-13T17:43:54.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.538+0000 7efd9599d640 1 -- 172.21.15.102:0/1586210509 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7efd9009c7d0 con 0x7efd90068920 2024-09-13T17:43:54.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.539+0000 7efd86ffd640 1 -- 172.21.15.102:0/1586210509 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7efd8002b080 con 0x7efd90068920 2024-09-13T17:43:54.541 INFO:teuthology.orchestra.run.smithi102.stderr:listed 0 entries 2024-09-13T17:43:54.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.541+0000 7efd9599d640 1 -- 172.21.15.102:0/1586210509 >> v1:172.21.15.102:6816/34937 conn(0x7efd70031160 legacy=0x7efd70033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:54.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.541+0000 7efd9599d640 1 -- 172.21.15.102:0/1586210509 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efd90068920 msgr2=0x7efd9011e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:54.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.541+0000 7efd9599d640 1 --2- 172.21.15.102:0/1586210509 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efd90068920 0x7efd9011e790 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efd8000c1a0 tx=0x7efd8000b040 comp rx=0 tx=0).stop 2024-09-13T17:43:54.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.541+0000 7efd9599d640 1 -- 172.21.15.102:0/1586210509 shutdown_connections 2024-09-13T17:43:54.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.541+0000 7efd9599d640 1 --2- 172.21.15.102:0/1586210509 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efd9009e020 0x7efd9011b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.541+0000 7efd9599d640 1 --2- 172.21.15.102:0/1586210509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd9009dc50 0x7efd9011af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.541+0000 7efd9599d640 1 --2- 172.21.15.102:0/1586210509 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efd90068920 0x7efd9011e790 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.541+0000 7efd9599d640 1 -- 172.21.15.102:0/1586210509 >> 172.21.15.102:0/1586210509 conn(0x7efd9006f320 msgr2=0x7efd901323f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:54.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.541+0000 7efd9599d640 1 -- 172.21.15.102:0/1586210509 shutdown_connections 2024-09-13T17:43:54.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.541+0000 7efd9599d640 1 -- 172.21.15.102:0/1586210509 wait complete. 2024-09-13T17:43:54.554 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi171.front.sepia.ceph.com... 2024-09-13T17:43:54.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:54.554 DEBUG:teuthology.orchestra.run.smithi171:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-13T17:43:54.585 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:43:54.586 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi171.front.sepia.ceph.com 2024-09-13T17:43:54.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:54.586 DEBUG:teuthology.orchestra.run.smithi171:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-13T17:43:54.643 INFO:tasks.cephfs.cephfs_test_case:[] 2024-09-13T17:43:54.644 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:43:54.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.736+0000 7fddee99d640 1 Processor -- start 2024-09-13T17:43:54.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.736+0000 7fddee99d640 1 -- start start 2024-09-13T17:43:54.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.736+0000 7fddee99d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fdde8128330 0x7fdde8128700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:54.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.736+0000 7fddee99d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdde812bf50 0x7fdde8133b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:54.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.736+0000 7fddee99d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdde8097be0 con 0x7fdde8128c40 2024-09-13T17:43:54.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.736+0000 7fddee99d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdde8098e70 con 0x7fdde8128330 2024-09-13T17:43:54.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.736+0000 7fddee99d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdde8098310 con 0x7fdde812bf50 2024-09-13T17:43:54.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.737+0000 7fdde7fff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fdde8128330 0x7fdde8128700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:54.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.737+0000 7fdde77fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fdde8128c40 0x7fdde812b840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:58920/0 (socket says 172.21.15.102:58920) 2024-09-13T17:43:54.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.737+0000 7fdde77fe640 1 -- 172.21.15.102:0/4181286982 learned_addr learned my addr 172.21.15.102:0/4181286982 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:54.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.737+0000 7fddecf3a640 1 --2- 172.21.15.102:0/4181286982 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdde812bf50 0x7fdde8133b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:54.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.737+0000 7fdde7fff640 1 -- 172.21.15.102:0/4181286982 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdde812bf50 msgr2=0x7fdde8133b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:54.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.737+0000 7fdde7fff640 1 --2- 172.21.15.102:0/4181286982 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdde812bf50 0x7fdde8133b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.737+0000 7fdde7fff640 1 -- 172.21.15.102:0/4181286982 >> v1:172.21.15.102:6789/0 conn(0x7fdde8128c40 legacy=0x7fdde812b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:54.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.737+0000 7fdde7fff640 1 -- 172.21.15.102:0/4181286982 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde80985d0 con 0x7fdde8128330 2024-09-13T17:43:54.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.737+0000 7fdde6ffd640 1 -- 172.21.15.102:0/4181286982 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2595892674 0 0) 0x7fdde8097be0 con 0x7fdde8128c40 2024-09-13T17:43:54.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.737+0000 7fddecf3a640 1 --2- 172.21.15.102:0/4181286982 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdde812bf50 0x7fdde8133b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:54.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.738+0000 7fdde7fff640 1 --2- 172.21.15.102:0/4181286982 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdde8128330 0x7fdde8128700 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fddd800baf0 tx=0x7fddd8006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305b2e8da32e924d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:54.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.738+0000 7fdde6ffd640 1 -- 172.21.15.102:0/4181286982 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd8032480 con 0x7fdde8128330 2024-09-13T17:43:54.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.738+0000 7fdde6ffd640 1 -- 172.21.15.102:0/4181286982 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd802f030 con 0x7fdde8128330 2024-09-13T17:43:54.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.738+0000 7fdde6ffd640 1 -- 172.21.15.102:0/4181286982 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd8032810 con 0x7fdde8128330 2024-09-13T17:43:54.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.738+0000 7fddee99d640 1 -- 172.21.15.102:0/4181286982 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdde8128330 msgr2=0x7fdde8128700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:54.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.738+0000 7fddee99d640 1 --2- 172.21.15.102:0/4181286982 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdde8128330 0x7fdde8128700 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fddd800baf0 tx=0x7fddd8006f90 comp rx=0 tx=0).stop 2024-09-13T17:43:54.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.738+0000 7fddee99d640 1 -- 172.21.15.102:0/4181286982 shutdown_connections 2024-09-13T17:43:54.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.738+0000 7fddee99d640 1 --2- 172.21.15.102:0/4181286982 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdde812bf50 0x7fdde8133b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.738+0000 7fddee99d640 1 --2- 172.21.15.102:0/4181286982 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdde8128330 0x7fdde8128700 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.738+0000 7fddee99d640 1 -- 172.21.15.102:0/4181286982 >> 172.21.15.102:0/4181286982 conn(0x7fdde809e820 msgr2=0x7fdde809ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:54.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.738+0000 7fddee99d640 1 -- 172.21.15.102:0/4181286982 shutdown_connections 2024-09-13T17:43:54.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.738+0000 7fddee99d640 1 -- 172.21.15.102:0/4181286982 wait complete. 2024-09-13T17:43:54.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.739+0000 7fddee99d640 1 Processor -- start 2024-09-13T17:43:54.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.739+0000 7fddee99d640 1 -- start start 2024-09-13T17:43:54.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.740+0000 7fddee99d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdde8128330 0x7fdde8114e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:54.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.740+0000 7fddee99d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdde8128c40 0x7fdde81153c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:54.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.740+0000 7fddee99d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdde812bf50 0x7fdde8115900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:54.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.740+0000 7fddee99d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde809a060 con 0x7fdde8128330 2024-09-13T17:43:54.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.740+0000 7fddee99d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdde8117280 con 0x7fdde8128c40 2024-09-13T17:43:54.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.740+0000 7fddee99d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdde8116fa0 con 0x7fdde812bf50 2024-09-13T17:43:54.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.740+0000 7fdde7fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdde8128330 0x7fdde8114e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:54.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.740+0000 7fdde77fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdde8128c40 0x7fdde81153c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:54.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.740+0000 7fddecf3a640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdde812bf50 0x7fdde8115900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:54.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.740+0000 7fdde7fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdde8128330 0x7fdde8114e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:46326/0 (socket says 172.21.15.102:46326) 2024-09-13T17:43:54.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.740+0000 7fdde7fff640 1 -- 172.21.15.102:0/2516324379 learned_addr learned my addr 172.21.15.102:0/2516324379 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:54.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.740+0000 7fdde7fff640 1 -- 172.21.15.102:0/2516324379 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdde812bf50 msgr2=0x7fdde8115900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:54.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.740+0000 7fdde7fff640 1 --2- 172.21.15.102:0/2516324379 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdde812bf50 0x7fdde8115900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.740+0000 7fdde7fff640 1 -- 172.21.15.102:0/2516324379 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdde8128c40 msgr2=0x7fdde81153c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:54.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.740+0000 7fdde7fff640 1 --2- 172.21.15.102:0/2516324379 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdde8128c40 0x7fdde81153c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.740+0000 7fdde7fff640 1 -- 172.21.15.102:0/2516324379 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde0009a20 con 0x7fdde8128330 2024-09-13T17:43:54.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.740+0000 7fdde7fff640 1 --2- 172.21.15.102:0/2516324379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdde8128330 0x7fdde8114e80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fddd802cb60 tx=0x7fddd803a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:54.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.740+0000 7fddcffff640 1 -- 172.21.15.102:0/2516324379 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd803e830 con 0x7fdde8128330 2024-09-13T17:43:54.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.740+0000 7fddcffff640 1 -- 172.21.15.102:0/2516324379 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd803e9b0 con 0x7fdde8128330 2024-09-13T17:43:54.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.740+0000 7fddee99d640 1 -- 172.21.15.102:0/2516324379 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd800b630 con 0x7fdde8128330 2024-09-13T17:43:54.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.740+0000 7fddcffff640 1 -- 172.21.15.102:0/2516324379 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd803ed70 con 0x7fdde8128330 2024-09-13T17:43:54.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.740+0000 7fddee99d640 1 -- 172.21.15.102:0/2516324379 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde806e9a0 con 0x7fdde8128330 2024-09-13T17:43:54.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.741+0000 7fddcffff640 1 -- 172.21.15.102:0/2516324379 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fddd8035820 con 0x7fdde8128330 2024-09-13T17:43:54.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.741+0000 7fddcffff640 1 -- 172.21.15.102:0/2516324379 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7fddd8066b30 con 0x7fdde8128330 2024-09-13T17:43:54.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.741+0000 7fddee99d640 1 -- 172.21.15.102:0/2516324379 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdde8064000 con 0x7fdde8128330 2024-09-13T17:43:54.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.746+0000 7fddcffff640 1 -- 172.21.15.102:0/2516324379 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fddd804d020 con 0x7fdde8128330 2024-09-13T17:43:54.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.926+0000 7fddee99d640 1 -- 172.21.15.102:0/2516324379 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdde8067690 con 0x7fdde8128330 2024-09-13T17:43:54.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.927+0000 7fddcffff640 1 -- 172.21.15.102:0/2516324379 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2266 (secure 0 0 0) 0x7fddd8056070 con 0x7fdde8128330 2024-09-13T17:43:54.929 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:43:54.929 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:42:06.611970+0000","modified":"2024-09-13T17:42:09.305093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4210":{"gid":4210,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6817/3443280887","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":3443280887}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-09-13T17:43:54.930 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 5 2024-09-13T17:43:54.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.929+0000 7fddee99d640 1 -- 172.21.15.102:0/2516324379 >> v1:172.21.15.102:6816/34937 conn(0x7fddd4031160 legacy=0x7fddd4033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:54.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.929+0000 7fddee99d640 1 -- 172.21.15.102:0/2516324379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdde8128330 msgr2=0x7fdde8114e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:54.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.929+0000 7fddee99d640 1 --2- 172.21.15.102:0/2516324379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdde8128330 0x7fdde8114e80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fddd802cb60 tx=0x7fddd803a000 comp rx=0 tx=0).stop 2024-09-13T17:43:54.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.929+0000 7fddee99d640 1 -- 172.21.15.102:0/2516324379 shutdown_connections 2024-09-13T17:43:54.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.929+0000 7fddee99d640 1 --2- 172.21.15.102:0/2516324379 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fdde812bf50 0x7fdde8115900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.929+0000 7fddee99d640 1 --2- 172.21.15.102:0/2516324379 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fdde8128c40 0x7fdde81153c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.929+0000 7fddee99d640 1 --2- 172.21.15.102:0/2516324379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdde8128330 0x7fdde8114e80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.929+0000 7fddee99d640 1 -- 172.21.15.102:0/2516324379 >> 172.21.15.102:0/2516324379 conn(0x7fdde809e820 msgr2=0x7fdde81261a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:54.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.929+0000 7fddee99d640 1 -- 172.21.15.102:0/2516324379 shutdown_connections 2024-09-13T17:43:54.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:54.929+0000 7fddee99d640 1 -- 172.21.15.102:0/2516324379 wait complete. 2024-09-13T17:43:54.941 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:43:55.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.033+0000 7fb6fd59d640 1 Processor -- start 2024-09-13T17:43:55.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.033+0000 7fb6fd59d640 1 -- start start 2024-09-13T17:43:55.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.033+0000 7fb6fd59d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb6f81305a0 0x7fb6f8132980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:55.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.033+0000 7fb6fd59d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb6f8132ec0 0x7fb6f81352a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:55.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.033+0000 7fb6fd59d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6f8099ec0 con 0x7fb6f812d590 2024-09-13T17:43:55.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.033+0000 7fb6fd59d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6f812d390 con 0x7fb6f8132ec0 2024-09-13T17:43:55.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.033+0000 7fb6fd59d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f809bc30 con 0x7fb6f81305a0 2024-09-13T17:43:55.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.033+0000 7fb6effff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb6f81305a0 0x7fb6f8132980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:55.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.034+0000 7fb6f6ffd640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fb6f812d590 0x7fb6f812d960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:58922/0 (socket says 172.21.15.102:58922) 2024-09-13T17:43:55.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.034+0000 7fb6f77fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb6f8132ec0 0x7fb6f81352a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:55.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.034+0000 7fb6f6ffd640 1 -- 172.21.15.102:0/3391143645 learned_addr learned my addr 172.21.15.102:0/3391143645 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:55.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.034+0000 7fb6effff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb6f81305a0 0x7fb6f8132980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:38720/0 (socket says 172.21.15.102:38720) 2024-09-13T17:43:55.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.034+0000 7fb6effff640 1 -- 172.21.15.102:0/3391143645 >> v2:172.21.15.102:3300/0 conn(0x7fb6f8132ec0 msgr2=0x7fb6f81352a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:55.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.034+0000 7fb6effff640 1 --2- 172.21.15.102:0/3391143645 >> v2:172.21.15.102:3300/0 conn(0x7fb6f8132ec0 0x7fb6f81352a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.034+0000 7fb6effff640 1 -- 172.21.15.102:0/3391143645 >> v1:172.21.15.102:6789/0 conn(0x7fb6f812d590 legacy=0x7fb6f812d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:55.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.034+0000 7fb6effff640 1 -- 172.21.15.102:0/3391143645 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f809aab0 con 0x7fb6f81305a0 2024-09-13T17:43:55.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.034+0000 7fb6f67fc640 1 -- 172.21.15.102:0/3391143645 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2025119235 0 0) 0x7fb6f8099ec0 con 0x7fb6f812d590 2024-09-13T17:43:55.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.034+0000 7fb6f77fe640 1 --2- 172.21.15.102:0/3391143645 >> v2:172.21.15.102:3300/0 conn(0x7fb6f8132ec0 0x7fb6f81352a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:55.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.034+0000 7fb6effff640 1 --2- 172.21.15.102:0/3391143645 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb6f81305a0 0x7fb6f8132980 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0009850 tx=0x7fb6e00090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ce13c697b07cf0c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:55.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.035+0000 7fb6f67fc640 1 -- 172.21.15.102:0/3391143645 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e002ad00 con 0x7fb6f81305a0 2024-09-13T17:43:55.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.035+0000 7fb6f67fc640 1 -- 172.21.15.102:0/3391143645 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e002ae60 con 0x7fb6f81305a0 2024-09-13T17:43:55.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.035+0000 7fb6f67fc640 1 -- 172.21.15.102:0/3391143645 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0032670 con 0x7fb6f81305a0 2024-09-13T17:43:55.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.035+0000 7fb6fd59d640 1 -- 172.21.15.102:0/3391143645 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb6f81305a0 msgr2=0x7fb6f8132980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:55.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.035+0000 7fb6fd59d640 1 --2- 172.21.15.102:0/3391143645 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb6f81305a0 0x7fb6f8132980 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0009850 tx=0x7fb6e00090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.035+0000 7fb6fd59d640 1 -- 172.21.15.102:0/3391143645 shutdown_connections 2024-09-13T17:43:55.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.035+0000 7fb6fd59d640 1 --2- 172.21.15.102:0/3391143645 >> v2:172.21.15.102:3300/0 conn(0x7fb6f8132ec0 0x7fb6f81352a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.035+0000 7fb6fd59d640 1 --2- 172.21.15.102:0/3391143645 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb6f81305a0 0x7fb6f8132980 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.035+0000 7fb6fd59d640 1 -- 172.21.15.102:0/3391143645 >> 172.21.15.102:0/3391143645 conn(0x7fb6f805db50 msgr2=0x7fb6f805df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:55.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.035+0000 7fb6fd59d640 1 -- 172.21.15.102:0/3391143645 shutdown_connections 2024-09-13T17:43:55.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.035+0000 7fb6fd59d640 1 -- 172.21.15.102:0/3391143645 wait complete. 2024-09-13T17:43:55.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.036+0000 7fb6fd59d640 1 Processor -- start 2024-09-13T17:43:55.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.036+0000 7fb6fd59d640 1 -- start start 2024-09-13T17:43:55.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.036+0000 7fb6fd59d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb6f812d590 0x7fb6f812cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:55.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.036+0000 7fb6fd59d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6f81305a0 0x7fb6f8096aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:55.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.036+0000 7fb6fd59d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb6f8132ec0 0x7fb6f8096fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:55.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.036+0000 7fb6fd59d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f8097f80 con 0x7fb6f81305a0 2024-09-13T17:43:55.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.036+0000 7fb6fd59d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f811c680 con 0x7fb6f8132ec0 2024-09-13T17:43:55.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.036+0000 7fb6fd59d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f80985d0 con 0x7fb6f812d590 2024-09-13T17:43:55.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.037+0000 7fb6f77fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb6f8132ec0 0x7fb6f8096fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:55.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.037+0000 7fb6f6ffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb6f812d590 0x7fb6f812cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:55.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.037+0000 7fb6f77fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb6f8132ec0 0x7fb6f8096fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:38730/0 (socket says 172.21.15.102:38730) 2024-09-13T17:43:55.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.037+0000 7fb6f77fe640 1 -- 172.21.15.102:0/4152827851 learned_addr learned my addr 172.21.15.102:0/4152827851 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:55.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.037+0000 7fb6effff640 1 --2- 172.21.15.102:0/4152827851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6f81305a0 0x7fb6f8096aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:55.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.037+0000 7fb6f77fe640 1 -- 172.21.15.102:0/4152827851 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb6f812d590 msgr2=0x7fb6f812cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:55.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.037+0000 7fb6f77fe640 1 --2- 172.21.15.102:0/4152827851 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb6f812d590 0x7fb6f812cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.037+0000 7fb6f77fe640 1 -- 172.21.15.102:0/4152827851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6f81305a0 msgr2=0x7fb6f8096aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:55.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.037+0000 7fb6f77fe640 1 --2- 172.21.15.102:0/4152827851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6f81305a0 0x7fb6f8096aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.037+0000 7fb6f77fe640 1 -- 172.21.15.102:0/4152827851 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f0009a20 con 0x7fb6f8132ec0 2024-09-13T17:43:55.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.037+0000 7fb6f6ffd640 1 --2- 172.21.15.102:0/4152827851 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb6f812d590 0x7fb6f812cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:55.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.037+0000 7fb6effff640 1 --2- 172.21.15.102:0/4152827851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6f81305a0 0x7fb6f8096aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:55.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.037+0000 7fb6f77fe640 1 --2- 172.21.15.102:0/4152827851 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb6f8132ec0 0x7fb6f8096fe0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb6f000efc0 tx=0x7fb6f000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:55.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.037+0000 7fb6eeffd640 1 -- 172.21.15.102:0/4152827851 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f000ce70 con 0x7fb6f8132ec0 2024-09-13T17:43:55.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.037+0000 7fb6eeffd640 1 -- 172.21.15.102:0/4152827851 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f000f040 con 0x7fb6f8132ec0 2024-09-13T17:43:55.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.037+0000 7fb6fd59d640 1 -- 172.21.15.102:0/4152827851 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e0009390 con 0x7fb6f8132ec0 2024-09-13T17:43:55.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.037+0000 7fb6eeffd640 1 -- 172.21.15.102:0/4152827851 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f00146b0 con 0x7fb6f8132ec0 2024-09-13T17:43:55.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.037+0000 7fb6fd59d640 1 -- 172.21.15.102:0/4152827851 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f811c070 con 0x7fb6f8132ec0 2024-09-13T17:43:55.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.038+0000 7fb6eeffd640 1 -- 172.21.15.102:0/4152827851 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fb6f0014aa0 con 0x7fb6f8132ec0 2024-09-13T17:43:55.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.038+0000 7fb6fd59d640 1 -- 172.21.15.102:0/4152827851 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6f8064000 con 0x7fb6f8132ec0 2024-09-13T17:43:55.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.038+0000 7fb6eeffd640 1 -- 172.21.15.102:0/4152827851 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7fb6f00430e0 con 0x7fb6f8132ec0 2024-09-13T17:43:55.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.043+0000 7fb6eeffd640 1 -- 172.21.15.102:0/4152827851 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fb6f0024530 con 0x7fb6f8132ec0 2024-09-13T17:43:55.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.195+0000 7fb6fd59d640 1 -- 172.21.15.102:0/4152827851 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6f811ae80 con 0x7fb6f8132ec0 2024-09-13T17:43:55.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.196+0000 7fb6eeffd640 1 -- 172.21.15.102:0/4152827851 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+11660 (secure 0 0 0) 0x7fb6f0013e00 con 0x7fb6f8132ec0 2024-09-13T17:43:55.198 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:43:55.198 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":23,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:42:48.385896+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:04.672265+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:05.583134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.17","mappings":[{"from":3,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]},{"pgid":"4.25","mappings":[{"from":3,"to":2}]},{"pgid":"4.33","mappings":[{"from":3,"to":0}]},{"pgid":"4.3a","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:43:55.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.198+0000 7fb6fd59d640 1 -- 172.21.15.102:0/4152827851 >> v1:172.21.15.102:6816/34937 conn(0x7fb6d8031160 legacy=0x7fb6d8033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:55.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.198+0000 7fb6fd59d640 1 -- 172.21.15.102:0/4152827851 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb6f8132ec0 msgr2=0x7fb6f8096fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:55.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.198+0000 7fb6fd59d640 1 --2- 172.21.15.102:0/4152827851 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb6f8132ec0 0x7fb6f8096fe0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb6f000efc0 tx=0x7fb6f000b040 comp rx=0 tx=0).stop 2024-09-13T17:43:55.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.198+0000 7fb6fd59d640 1 -- 172.21.15.102:0/4152827851 shutdown_connections 2024-09-13T17:43:55.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.198+0000 7fb6fd59d640 1 --2- 172.21.15.102:0/4152827851 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb6f8132ec0 0x7fb6f8096fe0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.198+0000 7fb6fd59d640 1 --2- 172.21.15.102:0/4152827851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6f81305a0 0x7fb6f8096aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.198+0000 7fb6fd59d640 1 --2- 172.21.15.102:0/4152827851 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb6f812d590 0x7fb6f812cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.198+0000 7fb6fd59d640 1 -- 172.21.15.102:0/4152827851 >> 172.21.15.102:0/4152827851 conn(0x7fb6f805db50 msgr2=0x7fb6f812f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:55.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.198+0000 7fb6fd59d640 1 -- 172.21.15.102:0/4152827851 shutdown_connections 2024-09-13T17:43:55.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.198+0000 7fb6fd59d640 1 -- 172.21.15.102:0/4152827851 wait complete. 2024-09-13T17:43:55.211 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:54.442612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T17:41:59.076782+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 16, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:42:04.672265+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:42:05.583134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:43:55.211 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:43:55.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.305+0000 7fc21cb09640 1 Processor -- start 2024-09-13T17:43:55.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.305+0000 7fc21cb09640 1 -- start start 2024-09-13T17:43:55.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.306+0000 7fc21cb09640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc21809f7f0 0x7fc2181325b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:55.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.306+0000 7fc21cb09640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc218132af0 0x7fc218134ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:55.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.306+0000 7fc21cb09640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc21806e210 con 0x7fc21806d670 2024-09-13T17:43:55.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.306+0000 7fc21cb09640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21806da40 con 0x7fc21809f7f0 2024-09-13T17:43:55.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.306+0000 7fc21cb09640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc21809fbc0 con 0x7fc218132af0 2024-09-13T17:43:55.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.306+0000 7fc216d9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc218132af0 0x7fc218134ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:55.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.306+0000 7fc215d9b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc21809f7f0 0x7fc2181325b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:55.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.306+0000 7fc21659c640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fc21806d670 0x7fc21806e780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:58928/0 (socket says 172.21.15.102:58928) 2024-09-13T17:43:55.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.306+0000 7fc21659c640 1 -- 172.21.15.102:0/3838091026 learned_addr learned my addr 172.21.15.102:0/3838091026 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:55.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.306+0000 7fc216d9d640 1 -- 172.21.15.102:0/3838091026 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc21809f7f0 msgr2=0x7fc2181325b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:55.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.306+0000 7fc216d9d640 1 --2- 172.21.15.102:0/3838091026 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc21809f7f0 0x7fc2181325b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.306+0000 7fc216d9d640 1 -- 172.21.15.102:0/3838091026 >> v1:172.21.15.102:6789/0 conn(0x7fc21806d670 legacy=0x7fc21806e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:55.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.306+0000 7fc216d9d640 1 -- 172.21.15.102:0/3838091026 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21809ff20 con 0x7fc218132af0 2024-09-13T17:43:55.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.306+0000 7fc21559a640 1 -- 172.21.15.102:0/3838091026 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3156081250 0 0) 0x7fc21806e210 con 0x7fc21806d670 2024-09-13T17:43:55.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.306+0000 7fc215d9b640 1 --2- 172.21.15.102:0/3838091026 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc21809f7f0 0x7fc2181325b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:55.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.307+0000 7fc216d9d640 1 --2- 172.21.15.102:0/3838091026 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc218132af0 0x7fc218134ed0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc210009880 tx=0x7fc2100090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6ffb00bbd01d05f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:55.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.307+0000 7fc21559a640 1 -- 172.21.15.102:0/3838091026 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21002ac50 con 0x7fc218132af0 2024-09-13T17:43:55.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.307+0000 7fc21559a640 1 -- 172.21.15.102:0/3838091026 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc21002adb0 con 0x7fc218132af0 2024-09-13T17:43:55.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.307+0000 7fc21559a640 1 -- 172.21.15.102:0/3838091026 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc210032670 con 0x7fc218132af0 2024-09-13T17:43:55.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.308+0000 7fc21cb09640 1 -- 172.21.15.102:0/3838091026 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc218132af0 msgr2=0x7fc218134ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:55.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.308+0000 7fc21cb09640 1 --2- 172.21.15.102:0/3838091026 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc218132af0 0x7fc218134ed0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc210009880 tx=0x7fc2100090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.308+0000 7fc21cb09640 1 -- 172.21.15.102:0/3838091026 shutdown_connections 2024-09-13T17:43:55.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.308+0000 7fc21cb09640 1 --2- 172.21.15.102:0/3838091026 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc218132af0 0x7fc218134ed0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.308+0000 7fc21cb09640 1 --2- 172.21.15.102:0/3838091026 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc21809f7f0 0x7fc2181325b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.308+0000 7fc21cb09640 1 -- 172.21.15.102:0/3838091026 >> 172.21.15.102:0/3838091026 conn(0x7fc218067bb0 msgr2=0x7fc2180a08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:55.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.308+0000 7fc21cb09640 1 -- 172.21.15.102:0/3838091026 shutdown_connections 2024-09-13T17:43:55.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.308+0000 7fc21cb09640 1 -- 172.21.15.102:0/3838091026 wait complete. 2024-09-13T17:43:55.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.309+0000 7fc21cb09640 1 Processor -- start 2024-09-13T17:43:55.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.309+0000 7fc21cb09640 1 -- start start 2024-09-13T17:43:55.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.309+0000 7fc21cb09640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc21806d670 0x7fc21809a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:55.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.309+0000 7fc21cb09640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc21809f7f0 0x7fc218068240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:55.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.309+0000 7fc21cb09640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc218132af0 0x7fc218068780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:55.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.309+0000 7fc21cb09640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc218069df0 con 0x7fc218132af0 2024-09-13T17:43:55.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.309+0000 7fc21cb09640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2180996b0 con 0x7fc21809f7f0 2024-09-13T17:43:55.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.309+0000 7fc21cb09640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc2181325d0 con 0x7fc21806d670 2024-09-13T17:43:55.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.309+0000 7fc21659c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc21806d670 0x7fc21809a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:55.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.309+0000 7fc215d9b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc21809f7f0 0x7fc218068240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:55.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.309+0000 7fc21659c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc21806d670 0x7fc21809a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:36322/0 (socket says 172.21.15.102:36322) 2024-09-13T17:43:55.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.309+0000 7fc21659c640 1 -- 172.21.15.102:0/2153791038 learned_addr learned my addr 172.21.15.102:0/2153791038 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:55.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.309+0000 7fc216d9d640 1 --2- 172.21.15.102:0/2153791038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc218132af0 0x7fc218068780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:55.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.309+0000 7fc21659c640 1 -- 172.21.15.102:0/2153791038 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc21809f7f0 msgr2=0x7fc218068240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:55.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.309+0000 7fc21659c640 1 --2- 172.21.15.102:0/2153791038 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc21809f7f0 0x7fc218068240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.309+0000 7fc21659c640 1 -- 172.21.15.102:0/2153791038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc218132af0 msgr2=0x7fc218068780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:55.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.309+0000 7fc21659c640 1 --2- 172.21.15.102:0/2153791038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc218132af0 0x7fc218068780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.309+0000 7fc21659c640 1 -- 172.21.15.102:0/2153791038 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc200009390 con 0x7fc21806d670 2024-09-13T17:43:55.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.310+0000 7fc21659c640 1 --2- 172.21.15.102:0/2153791038 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc21806d670 0x7fc21809a4c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc20800ccf0 tx=0x7fc208001a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:55.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.310+0000 7fc20e7fc640 1 -- 172.21.15.102:0/2153791038 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20800a850 con 0x7fc21806d670 2024-09-13T17:43:55.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.310+0000 7fc20e7fc640 1 -- 172.21.15.102:0/2153791038 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20800a9b0 con 0x7fc21806d670 2024-09-13T17:43:55.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.310+0000 7fc21cb09640 1 -- 172.21.15.102:0/2153791038 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc210009390 con 0x7fc21806d670 2024-09-13T17:43:55.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.310+0000 7fc20e7fc640 1 -- 172.21.15.102:0/2153791038 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20800ad90 con 0x7fc21806d670 2024-09-13T17:43:55.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.310+0000 7fc21cb09640 1 -- 172.21.15.102:0/2153791038 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2180989d0 con 0x7fc21806d670 2024-09-13T17:43:55.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.311+0000 7fc20e7fc640 1 -- 172.21.15.102:0/2153791038 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fc208007300 con 0x7fc21806d670 2024-09-13T17:43:55.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.311+0000 7fc20e7fc640 1 -- 172.21.15.102:0/2153791038 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7fc208042280 con 0x7fc21806d670 2024-09-13T17:43:55.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.311+0000 7fc21cb09640 1 -- 172.21.15.102:0/2153791038 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc218064600 con 0x7fc21806d670 2024-09-13T17:43:55.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.315+0000 7fc20e7fc640 1 -- 172.21.15.102:0/2153791038 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fc20801bae0 con 0x7fc21806d670 2024-09-13T17:43:55.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.467+0000 7fc21cb09640 1 -- 172.21.15.102:0/2153791038 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc21809af10 con 0x7fc21806d670 2024-09-13T17:43:55.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.467+0000 7fc20e7fc640 1 -- 172.21.15.102:0/2153791038 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc208010aa0 con 0x7fc21806d670 2024-09-13T17:43:55.469 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-09-13T17:43:55.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.469+0000 7fc21cb09640 1 -- 172.21.15.102:0/2153791038 >> v1:172.21.15.102:6816/34937 conn(0x7fc1f8031160 legacy=0x7fc1f8033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:55.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.469+0000 7fc21cb09640 1 -- 172.21.15.102:0/2153791038 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc21806d670 msgr2=0x7fc21809a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:55.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.469+0000 7fc21cb09640 1 --2- 172.21.15.102:0/2153791038 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc21806d670 0x7fc21809a4c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc20800ccf0 tx=0x7fc208001a80 comp rx=0 tx=0).stop 2024-09-13T17:43:55.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.469+0000 7fc21cb09640 1 -- 172.21.15.102:0/2153791038 shutdown_connections 2024-09-13T17:43:55.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.469+0000 7fc21cb09640 1 --2- 172.21.15.102:0/2153791038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc218132af0 0x7fc218068780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.469+0000 7fc21cb09640 1 --2- 172.21.15.102:0/2153791038 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc21809f7f0 0x7fc218068240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.469+0000 7fc21cb09640 1 --2- 172.21.15.102:0/2153791038 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc21806d670 0x7fc21809a4c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.469+0000 7fc21cb09640 1 -- 172.21.15.102:0/2153791038 >> 172.21.15.102:0/2153791038 conn(0x7fc218067bb0 msgr2=0x7fc21809daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:55.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.469+0000 7fc21cb09640 1 -- 172.21.15.102:0/2153791038 shutdown_connections 2024-09-13T17:43:55.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.469+0000 7fc21cb09640 1 -- 172.21.15.102:0/2153791038 wait complete. 2024-09-13T17:43:55.482 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-13T17:43:55.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.577+0000 7fc86c09f640 1 Processor -- start 2024-09-13T17:43:55.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.577+0000 7fc86c09f640 1 -- start start 2024-09-13T17:43:55.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.578+0000 7fc86c09f640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc864096a70 0x7fc8640987c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:55.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.578+0000 7fc86c09f640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc864098d00 0x7fc86412c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:55.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.578+0000 7fc86c09f640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc864099110 0x7fc86412c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:55.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.578+0000 7fc86c09f640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc864069c30 con 0x7fc864099110 2024-09-13T17:43:55.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.578+0000 7fc86c09f640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc86406ed00 con 0x7fc864098d00 2024-09-13T17:43:55.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.578+0000 7fc86c09f640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc8640999e0 con 0x7fc864096a70 2024-09-13T17:43:55.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.578+0000 7fc86963a640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc864098d00 0x7fc86412c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:55.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.578+0000 7fc86a63c640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc864099110 0x7fc86412c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:55.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.578+0000 7fc869e3b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc864096a70 0x7fc8640987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:55.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.578+0000 7fc86963a640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc864098d00 0x7fc86412c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:38746/0 (socket says 172.21.15.102:38746) 2024-09-13T17:43:55.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.578+0000 7fc86963a640 1 -- 172.21.15.102:0/3720724567 learned_addr learned my addr 172.21.15.102:0/3720724567 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:55.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.578+0000 7fc86a63c640 1 -- 172.21.15.102:0/3720724567 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc864096a70 msgr2=0x7fc8640987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:55.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.578+0000 7fc86a63c640 1 --2- 172.21.15.102:0/3720724567 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc864096a70 0x7fc8640987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.578+0000 7fc86a63c640 1 -- 172.21.15.102:0/3720724567 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc864098d00 msgr2=0x7fc86412c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:55.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.578+0000 7fc86a63c640 1 --2- 172.21.15.102:0/3720724567 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc864098d00 0x7fc86412c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.578+0000 7fc86a63c640 1 -- 172.21.15.102:0/3720724567 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8640997a0 con 0x7fc864099110 2024-09-13T17:43:55.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.578+0000 7fc869e3b640 1 --2- 172.21.15.102:0/3720724567 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc864096a70 0x7fc8640987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:55.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.578+0000 7fc86963a640 1 --2- 172.21.15.102:0/3720724567 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc864098d00 0x7fc86412c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:55.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.579+0000 7fc86a63c640 1 --2- 172.21.15.102:0/3720724567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc864099110 0x7fc86412c820 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc85c009e90 tx=0x7fc85c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3071a272272ed39 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:55.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.579+0000 7fc868e39640 1 -- 172.21.15.102:0/3720724567 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c031400 con 0x7fc864099110 2024-09-13T17:43:55.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.579+0000 7fc868e39640 1 -- 172.21.15.102:0/3720724567 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85c02d050 con 0x7fc864099110 2024-09-13T17:43:55.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.579+0000 7fc868e39640 1 -- 172.21.15.102:0/3720724567 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c0317d0 con 0x7fc864099110 2024-09-13T17:43:55.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.579+0000 7fc86c09f640 1 -- 172.21.15.102:0/3720724567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc864099110 msgr2=0x7fc86412c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:55.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.579+0000 7fc86c09f640 1 --2- 172.21.15.102:0/3720724567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc864099110 0x7fc86412c820 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc85c009e90 tx=0x7fc85c009780 comp rx=0 tx=0).stop 2024-09-13T17:43:55.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.579+0000 7fc86c09f640 1 -- 172.21.15.102:0/3720724567 shutdown_connections 2024-09-13T17:43:55.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.579+0000 7fc86c09f640 1 --2- 172.21.15.102:0/3720724567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc864099110 0x7fc86412c820 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.579+0000 7fc86c09f640 1 --2- 172.21.15.102:0/3720724567 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc864098d00 0x7fc86412c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.579+0000 7fc86c09f640 1 --2- 172.21.15.102:0/3720724567 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc864096a70 0x7fc8640987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.579+0000 7fc86c09f640 1 -- 172.21.15.102:0/3720724567 >> 172.21.15.102:0/3720724567 conn(0x7fc86409e720 msgr2=0x7fc86409eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:55.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.580+0000 7fc86c09f640 1 -- 172.21.15.102:0/3720724567 shutdown_connections 2024-09-13T17:43:55.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.580+0000 7fc86c09f640 1 -- 172.21.15.102:0/3720724567 wait complete. 2024-09-13T17:43:55.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.580+0000 7fc86c09f640 1 Processor -- start 2024-09-13T17:43:55.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.580+0000 7fc86c09f640 1 -- start start 2024-09-13T17:43:55.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.581+0000 7fc86c09f640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc864096a70 0x7fc86412f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:55.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.581+0000 7fc86c09f640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc864098d00 0x7fc86412f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:55.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.581+0000 7fc86c09f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc864099110 0x7fc86412ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:55.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.581+0000 7fc86c09f640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86409a410 con 0x7fc864099110 2024-09-13T17:43:55.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.581+0000 7fc86c09f640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8641328e0 con 0x7fc864096a70 2024-09-13T17:43:55.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.581+0000 7fc86c09f640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc86412a1e0 con 0x7fc864098d00 2024-09-13T17:43:55.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.581+0000 7fc86963a640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc864098d00 0x7fc86412f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:55.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.581+0000 7fc869e3b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc864096a70 0x7fc86412f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:55.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.581+0000 7fc869e3b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc864096a70 0x7fc86412f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:38754/0 (socket says 172.21.15.102:38754) 2024-09-13T17:43:55.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.581+0000 7fc869e3b640 1 -- 172.21.15.102:0/386002957 learned_addr learned my addr 172.21.15.102:0/386002957 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:55.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.581+0000 7fc86a63c640 1 --2- 172.21.15.102:0/386002957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc864099110 0x7fc86412ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:55.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.581+0000 7fc869e3b640 1 -- 172.21.15.102:0/386002957 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc864098d00 msgr2=0x7fc86412f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:55.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.581+0000 7fc869e3b640 1 --2- 172.21.15.102:0/386002957 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc864098d00 0x7fc86412f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.581+0000 7fc869e3b640 1 -- 172.21.15.102:0/386002957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc864099110 msgr2=0x7fc86412ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:55.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.581+0000 7fc869e3b640 1 --2- 172.21.15.102:0/386002957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc864099110 0x7fc86412ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.581+0000 7fc869e3b640 1 -- 172.21.15.102:0/386002957 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc860009390 con 0x7fc864096a70 2024-09-13T17:43:55.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.581+0000 7fc86963a640 1 --2- 172.21.15.102:0/386002957 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc864098d00 0x7fc86412f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:55.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.581+0000 7fc86a63c640 1 --2- 172.21.15.102:0/386002957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc864099110 0x7fc86412ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:55.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.581+0000 7fc869e3b640 1 --2- 172.21.15.102:0/386002957 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc864096a70 0x7fc86412f490 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc86000ec80 tx=0x7fc8600090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:55.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.581+0000 7fc859ffb640 1 -- 172.21.15.102:0/386002957 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86000eec0 con 0x7fc864096a70 2024-09-13T17:43:55.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.581+0000 7fc859ffb640 1 -- 172.21.15.102:0/386002957 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86000cbc0 con 0x7fc864096a70 2024-09-13T17:43:55.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.582+0000 7fc86c09f640 1 -- 172.21.15.102:0/386002957 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc854009450 con 0x7fc864096a70 2024-09-13T17:43:55.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.582+0000 7fc859ffb640 1 -- 172.21.15.102:0/386002957 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc860013470 con 0x7fc864096a70 2024-09-13T17:43:55.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.582+0000 7fc86c09f640 1 -- 172.21.15.102:0/386002957 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc85c009a20 con 0x7fc864096a70 2024-09-13T17:43:55.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.582+0000 7fc859ffb640 1 -- 172.21.15.102:0/386002957 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fc8600137e0 con 0x7fc864096a70 2024-09-13T17:43:55.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.583+0000 7fc859ffb640 1 -- 172.21.15.102:0/386002957 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7fc860042530 con 0x7fc864096a70 2024-09-13T17:43:55.588 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.583+0000 7fc86c09f640 1 -- 172.21.15.102:0/386002957 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc864131030 con 0x7fc864096a70 2024-09-13T17:43:55.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.587+0000 7fc859ffb640 1 -- 172.21.15.102:0/386002957 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fc860021020 con 0x7fc864096a70 2024-09-13T17:43:55.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.742+0000 7fc86c09f640 1 -- 172.21.15.102:0/386002957 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc864132b00 con 0x7fc864096a70 2024-09-13T17:43:55.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.742+0000 7fc859ffb640 1 -- 172.21.15.102:0/386002957 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc860023ad0 con 0x7fc864096a70 2024-09-13T17:43:55.745 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-09-13T17:43:55.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.744+0000 7fc86c09f640 1 -- 172.21.15.102:0/386002957 >> v1:172.21.15.102:6816/34937 conn(0x7fc848031160 legacy=0x7fc848033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:55.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.744+0000 7fc86c09f640 1 -- 172.21.15.102:0/386002957 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc864096a70 msgr2=0x7fc86412f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:55.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.744+0000 7fc86c09f640 1 --2- 172.21.15.102:0/386002957 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc864096a70 0x7fc86412f490 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc86000ec80 tx=0x7fc8600090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.744+0000 7fc86c09f640 1 -- 172.21.15.102:0/386002957 shutdown_connections 2024-09-13T17:43:55.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.744+0000 7fc86c09f640 1 --2- 172.21.15.102:0/386002957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc864099110 0x7fc86412ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.744+0000 7fc86c09f640 1 --2- 172.21.15.102:0/386002957 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc864098d00 0x7fc86412f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.744+0000 7fc86c09f640 1 --2- 172.21.15.102:0/386002957 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc864096a70 0x7fc86412f490 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.744+0000 7fc86c09f640 1 -- 172.21.15.102:0/386002957 >> 172.21.15.102:0/386002957 conn(0x7fc86409e720 msgr2=0x7fc86409d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:55.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.744+0000 7fc86c09f640 1 -- 172.21.15.102:0/386002957 shutdown_connections 2024-09-13T17:43:55.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.744+0000 7fc86c09f640 1 -- 172.21.15.102:0/386002957 wait complete. 2024-09-13T17:43:55.757 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:43:55.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.851+0000 7f99e795d640 1 Processor -- start 2024-09-13T17:43:55.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.851+0000 7f99e795d640 1 -- start start 2024-09-13T17:43:55.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.851+0000 7f99e795d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f99e0096a70 0x7f99e012c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:55.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.851+0000 7f99e795d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f99e012c900 0x7f99e0098410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:55.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.851+0000 7f99e795d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f99e012ccd0 0x7f99e0098950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:55.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.851+0000 7f99e795d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99e0096e40 con 0x7f99e012c900 2024-09-13T17:43:55.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.851+0000 7f99e795d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e0099540 con 0x7f99e0096a70 2024-09-13T17:43:55.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.851+0000 7f99e795d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f99e009a8e0 con 0x7f99e012ccd0 2024-09-13T17:43:55.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.851+0000 7f99e56f9640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f99e0096a70 0x7f99e012c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:55.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.851+0000 7f99e4ef8640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f99e012c900 0x7f99e0098410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:55.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.851+0000 7f99e5efa640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f99e012ccd0 0x7f99e0098950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:55.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.851+0000 7f99e56f9640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f99e0096a70 0x7f99e012c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:38770/0 (socket says 172.21.15.102:38770) 2024-09-13T17:43:55.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.851+0000 7f99e4ef8640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f99e012c900 0x7f99e0098410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:46374/0 (socket says 172.21.15.102:46374) 2024-09-13T17:43:55.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.851+0000 7f99e56f9640 1 -- 172.21.15.102:0/3923981069 learned_addr learned my addr 172.21.15.102:0/3923981069 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:55.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.851+0000 7f99e5efa640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f99e012ccd0 0x7f99e0098950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:36348/0 (socket says 172.21.15.102:36348) 2024-09-13T17:43:55.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.852+0000 7f99e56f9640 1 -- 172.21.15.102:0/3923981069 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f99e012ccd0 msgr2=0x7f99e0098950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:55.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.852+0000 7f99e56f9640 1 --2- 172.21.15.102:0/3923981069 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f99e012ccd0 0x7f99e0098950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.852+0000 7f99e56f9640 1 -- 172.21.15.102:0/3923981069 >> v2:172.21.15.102:3300/0 conn(0x7f99e012c900 msgr2=0x7f99e0098410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:55.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.852+0000 7f99e56f9640 1 --2- 172.21.15.102:0/3923981069 >> v2:172.21.15.102:3300/0 conn(0x7f99e012c900 0x7f99e0098410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.852+0000 7f99e56f9640 1 -- 172.21.15.102:0/3923981069 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e0099760 con 0x7f99e0096a70 2024-09-13T17:43:55.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.852+0000 7f99e5efa640 1 --2- 172.21.15.102:0/3923981069 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f99e012ccd0 0x7f99e0098950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:55.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.852+0000 7f99e4ef8640 1 --2- 172.21.15.102:0/3923981069 >> v2:172.21.15.102:3300/0 conn(0x7f99e012c900 0x7f99e0098410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:55.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.852+0000 7f99e56f9640 1 --2- 172.21.15.102:0/3923981069 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f99e0096a70 0x7f99e012c3c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f99dc009940 tx=0x7f99dc0091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afc3330d318f2a3c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:55.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.853+0000 7f99d7fff640 1 -- 172.21.15.102:0/3923981069 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99dc034440 con 0x7f99e0096a70 2024-09-13T17:43:55.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.853+0000 7f99d7fff640 1 -- 172.21.15.102:0/3923981069 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99dc0345a0 con 0x7f99e0096a70 2024-09-13T17:43:55.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.853+0000 7f99d7fff640 1 -- 172.21.15.102:0/3923981069 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99dc034980 con 0x7f99e0096a70 2024-09-13T17:43:55.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.853+0000 7f99e795d640 1 -- 172.21.15.102:0/3923981069 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f99e0096a70 msgr2=0x7f99e012c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:55.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.853+0000 7f99e795d640 1 --2- 172.21.15.102:0/3923981069 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f99e0096a70 0x7f99e012c3c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f99dc009940 tx=0x7f99dc0091d0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.853+0000 7f99e795d640 1 -- 172.21.15.102:0/3923981069 shutdown_connections 2024-09-13T17:43:55.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.853+0000 7f99e795d640 1 --2- 172.21.15.102:0/3923981069 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f99e012ccd0 0x7f99e0098950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.853+0000 7f99e795d640 1 --2- 172.21.15.102:0/3923981069 >> v2:172.21.15.102:3300/0 conn(0x7f99e012c900 0x7f99e0098410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.853+0000 7f99e795d640 1 --2- 172.21.15.102:0/3923981069 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f99e0096a70 0x7f99e012c3c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.853+0000 7f99e795d640 1 -- 172.21.15.102:0/3923981069 >> 172.21.15.102:0/3923981069 conn(0x7f99e009e6b0 msgr2=0x7f99e009eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:55.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.853+0000 7f99e795d640 1 -- 172.21.15.102:0/3923981069 shutdown_connections 2024-09-13T17:43:55.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.853+0000 7f99e795d640 1 -- 172.21.15.102:0/3923981069 wait complete. 2024-09-13T17:43:55.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.854+0000 7f99e795d640 1 Processor -- start 2024-09-13T17:43:55.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.854+0000 7f99e795d640 1 -- start start 2024-09-13T17:43:55.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.855+0000 7f99e795d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f99e0096a70 0x7f99e013a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:55.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.855+0000 7f99e795d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99e012c900 0x7f99e013ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:55.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.855+0000 7f99e795d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f99e012ccd0 0x7f99e0115290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:55.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.855+0000 7f99e795d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e0118720 con 0x7f99e012c900 2024-09-13T17:43:55.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.855+0000 7f99e795d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e0117210 con 0x7f99e012ccd0 2024-09-13T17:43:55.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.855+0000 7f99e795d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f99e0118900 con 0x7f99e0096a70 2024-09-13T17:43:55.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.855+0000 7f99e56f9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f99e0096a70 0x7f99e013a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:55.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.855+0000 7f99e4ef8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99e012c900 0x7f99e013ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:55.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.855+0000 7f99e56f9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f99e0096a70 0x7f99e013a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:36354/0 (socket says 172.21.15.102:36354) 2024-09-13T17:43:55.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.855+0000 7f99e5efa640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f99e012ccd0 0x7f99e0115290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:55.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.855+0000 7f99e56f9640 1 -- 172.21.15.102:0/1802320223 learned_addr learned my addr 172.21.15.102:0/1802320223 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:55.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.855+0000 7f99e56f9640 1 -- 172.21.15.102:0/1802320223 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f99e012ccd0 msgr2=0x7f99e0115290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:55.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.855+0000 7f99e56f9640 1 --2- 172.21.15.102:0/1802320223 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f99e012ccd0 0x7f99e0115290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.855+0000 7f99e56f9640 1 -- 172.21.15.102:0/1802320223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99e012c900 msgr2=0x7f99e013ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:55.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.855+0000 7f99e56f9640 1 --2- 172.21.15.102:0/1802320223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99e012c900 0x7f99e013ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.855+0000 7f99e56f9640 1 -- 172.21.15.102:0/1802320223 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d0009390 con 0x7f99e0096a70 2024-09-13T17:43:55.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.855+0000 7f99e5efa640 1 --2- 172.21.15.102:0/1802320223 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f99e012ccd0 0x7f99e0115290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:55.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.855+0000 7f99e4ef8640 1 --2- 172.21.15.102:0/1802320223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99e012c900 0x7f99e013ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:55.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.855+0000 7f99e56f9640 1 --2- 172.21.15.102:0/1802320223 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f99e0096a70 0x7f99e013a820 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f99dc037040 tx=0x7f99dc034000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:55.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.855+0000 7f99d57fa640 1 -- 172.21.15.102:0/1802320223 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99dc033a60 con 0x7f99e0096a70 2024-09-13T17:43:55.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.855+0000 7f99d57fa640 1 -- 172.21.15.102:0/1802320223 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99dc033c00 con 0x7f99e0096a70 2024-09-13T17:43:55.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.855+0000 7f99e795d640 1 -- 172.21.15.102:0/1802320223 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99d8009a20 con 0x7f99e0096a70 2024-09-13T17:43:55.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.855+0000 7f99d57fa640 1 -- 172.21.15.102:0/1802320223 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99dc03f440 con 0x7f99e0096a70 2024-09-13T17:43:55.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.855+0000 7f99e795d640 1 -- 172.21.15.102:0/1802320223 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99dc009450 con 0x7f99e0096a70 2024-09-13T17:43:55.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.856+0000 7f99d57fa640 1 -- 172.21.15.102:0/1802320223 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f99dc03b040 con 0x7f99e0096a70 2024-09-13T17:43:55.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.856+0000 7f99d57fa640 1 -- 172.21.15.102:0/1802320223 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f99dc063c40 con 0x7f99e0096a70 2024-09-13T17:43:55.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.857+0000 7f99e795d640 1 -- 172.21.15.102:0/1802320223 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99e0064000 con 0x7f99e0096a70 2024-09-13T17:43:55.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:55.861+0000 7f99d57fa640 1 -- 172.21.15.102:0/1802320223 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f99dc039020 con 0x7f99e0096a70 2024-09-13T17:43:56.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.010+0000 7f99e795d640 1 -- 172.21.15.102:0/1802320223 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f99e0116cd0 con 0x7f99e0096a70 2024-09-13T17:43:56.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.010+0000 7f99d57fa640 1 -- 172.21.15.102:0/1802320223 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f99dc046af0 con 0x7f99e0096a70 2024-09-13T17:43:56.012 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-09-13T17:43:56.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.012+0000 7f99e795d640 1 -- 172.21.15.102:0/1802320223 >> v1:172.21.15.102:6816/34937 conn(0x7f99c4031160 legacy=0x7f99c4033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.012+0000 7f99e795d640 1 -- 172.21.15.102:0/1802320223 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f99e0096a70 msgr2=0x7f99e013a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.012+0000 7f99e795d640 1 --2- 172.21.15.102:0/1802320223 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f99e0096a70 0x7f99e013a820 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f99dc037040 tx=0x7f99dc034000 comp rx=0 tx=0).stop 2024-09-13T17:43:56.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.012+0000 7f99e795d640 1 -- 172.21.15.102:0/1802320223 shutdown_connections 2024-09-13T17:43:56.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.012+0000 7f99e795d640 1 --2- 172.21.15.102:0/1802320223 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f99e012ccd0 0x7f99e0115290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.012+0000 7f99e795d640 1 --2- 172.21.15.102:0/1802320223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99e012c900 0x7f99e013ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.012+0000 7f99e795d640 1 --2- 172.21.15.102:0/1802320223 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f99e0096a70 0x7f99e013a820 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.012+0000 7f99e795d640 1 -- 172.21.15.102:0/1802320223 >> 172.21.15.102:0/1802320223 conn(0x7f99e009e6b0 msgr2=0x7f99e012d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:56.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.012+0000 7f99e795d640 1 -- 172.21.15.102:0/1802320223 shutdown_connections 2024-09-13T17:43:56.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.012+0000 7f99e795d640 1 -- 172.21.15.102:0/1802320223 wait complete. 2024-09-13T17:43:56.025 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:43:56.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.120+0000 7f19b5620640 1 Processor -- start 2024-09-13T17:43:56.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.120+0000 7f19b5620640 1 -- start start 2024-09-13T17:43:56.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.120+0000 7f19b5620640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f19b00689d0 0x7f19b009d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:56.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.120+0000 7f19b5620640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f19b01348a0 0x7f19b0136c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:56.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.120+0000 7f19b5620640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19b009f4b0 con 0x7f19b00698b0 2024-09-13T17:43:56.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.120+0000 7f19b5620640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19b0096d80 con 0x7f19b00689d0 2024-09-13T17:43:56.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.120+0000 7f19b5620640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f19b009e420 con 0x7f19b01348a0 2024-09-13T17:43:56.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.120+0000 7f19af7fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f19b01348a0 0x7f19b0136c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:56.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.120+0000 7f19aeffd640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f19b00689d0 0x7f19b009d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:56.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.120+0000 7f19a67fc640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f19b00698b0 0x7f19b009dd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:58932/0 (socket says 172.21.15.102:58932) 2024-09-13T17:43:56.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.120+0000 7f19af7fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f19b01348a0 0x7f19b0136c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:36368/0 (socket says 172.21.15.102:36368) 2024-09-13T17:43:56.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.120+0000 7f19a67fc640 1 -- 172.21.15.102:0/1019024261 learned_addr learned my addr 172.21.15.102:0/1019024261 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:56.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.121+0000 7f19aeffd640 1 -- 172.21.15.102:0/1019024261 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f19b01348a0 msgr2=0x7f19b0136c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:56.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.121+0000 7f19aeffd640 1 --2- 172.21.15.102:0/1019024261 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f19b01348a0 0x7f19b0136c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.121+0000 7f19aeffd640 1 -- 172.21.15.102:0/1019024261 >> v1:172.21.15.102:6789/0 conn(0x7f19b00698b0 legacy=0x7f19b009dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.121+0000 7f19aeffd640 1 -- 172.21.15.102:0/1019024261 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b009cf10 con 0x7f19b00689d0 2024-09-13T17:43:56.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.121+0000 7f19af7fe640 1 --2- 172.21.15.102:0/1019024261 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f19b01348a0 0x7f19b0136c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:56.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.121+0000 7f19aeffd640 1 --2- 172.21.15.102:0/1019024261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19b00689d0 0x7f19b009d7d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009850 tx=0x7f19a00090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2ef926631471e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:56.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.121+0000 7f19ae7fc640 1 -- 172.21.15.102:0/1019024261 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a002ac80 con 0x7f19b00689d0 2024-09-13T17:43:56.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.122+0000 7f19ae7fc640 1 -- 172.21.15.102:0/1019024261 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a002ade0 con 0x7f19b00689d0 2024-09-13T17:43:56.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.122+0000 7f19ae7fc640 1 -- 172.21.15.102:0/1019024261 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a0032670 con 0x7f19b00689d0 2024-09-13T17:43:56.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.122+0000 7f19b5620640 1 -- 172.21.15.102:0/1019024261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19b00689d0 msgr2=0x7f19b009d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.122+0000 7f19b5620640 1 --2- 172.21.15.102:0/1019024261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19b00689d0 0x7f19b009d7d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009850 tx=0x7f19a00090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.122+0000 7f19b5620640 1 -- 172.21.15.102:0/1019024261 shutdown_connections 2024-09-13T17:43:56.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.122+0000 7f19b5620640 1 --2- 172.21.15.102:0/1019024261 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f19b01348a0 0x7f19b0136c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.122+0000 7f19b5620640 1 --2- 172.21.15.102:0/1019024261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19b00689d0 0x7f19b009d7d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.122+0000 7f19b5620640 1 -- 172.21.15.102:0/1019024261 >> 172.21.15.102:0/1019024261 conn(0x7f19b00a2bd0 msgr2=0x7f19b00a2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:56.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.122+0000 7f19b5620640 1 -- 172.21.15.102:0/1019024261 shutdown_connections 2024-09-13T17:43:56.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.122+0000 7f19b5620640 1 -- 172.21.15.102:0/1019024261 wait complete. 2024-09-13T17:43:56.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.123+0000 7f19b5620640 1 Processor -- start 2024-09-13T17:43:56.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.123+0000 7f19b5620640 1 -- start start 2024-09-13T17:43:56.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.123+0000 7f19b5620640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f19b00689d0 0x7f19b012cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:56.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.123+0000 7f19b5620640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f19b00698b0 0x7f19b012ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:56.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.123+0000 7f19b5620640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19b01348a0 0x7f19b01304a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:56.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.123+0000 7f19b5620640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b01328b0 con 0x7f19b01348a0 2024-09-13T17:43:56.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.123+0000 7f19b5620640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19b0131ee0 con 0x7f19b00689d0 2024-09-13T17:43:56.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.123+0000 7f19b5620640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f19b0132120 con 0x7f19b00698b0 2024-09-13T17:43:56.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.124+0000 7f19aeffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f19b00689d0 0x7f19b012cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:56.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.124+0000 7f19a67fc640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f19b00698b0 0x7f19b012ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:56.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.124+0000 7f19af7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19b01348a0 0x7f19b01304a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:56.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.124+0000 7f19aeffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f19b00689d0 0x7f19b012cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:38790/0 (socket says 172.21.15.102:38790) 2024-09-13T17:43:56.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.124+0000 7f19aeffd640 1 -- 172.21.15.102:0/3224247864 learned_addr learned my addr 172.21.15.102:0/3224247864 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:56.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.124+0000 7f19af7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19b01348a0 0x7f19b01304a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:46398/0 (socket says 172.21.15.102:46398) 2024-09-13T17:43:56.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.124+0000 7f19af7fe640 1 -- 172.21.15.102:0/3224247864 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f19b00698b0 msgr2=0x7f19b012ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.124+0000 7f19af7fe640 1 --2- 172.21.15.102:0/3224247864 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f19b00698b0 0x7f19b012ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.124+0000 7f19af7fe640 1 -- 172.21.15.102:0/3224247864 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f19b00689d0 msgr2=0x7f19b012cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.124+0000 7f19af7fe640 1 --2- 172.21.15.102:0/3224247864 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f19b00689d0 0x7f19b012cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.124+0000 7f19af7fe640 1 -- 172.21.15.102:0/3224247864 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a8009a20 con 0x7f19b01348a0 2024-09-13T17:43:56.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.124+0000 7f19a67fc640 1 --2- 172.21.15.102:0/3224247864 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f19b00698b0 0x7f19b012ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:56.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.124+0000 7f19af7fe640 1 --2- 172.21.15.102:0/3224247864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19b01348a0 0x7f19b01304a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f19a8009f40 tx=0x7f19a800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:56.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.124+0000 7f19a77fe640 1 -- 172.21.15.102:0/3224247864 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a800ce10 con 0x7f19b01348a0 2024-09-13T17:43:56.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.124+0000 7f19a77fe640 1 -- 172.21.15.102:0/3224247864 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a800f040 con 0x7f19b01348a0 2024-09-13T17:43:56.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.124+0000 7f19b5620640 1 -- 172.21.15.102:0/3224247864 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19a0009390 con 0x7f19b01348a0 2024-09-13T17:43:56.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.124+0000 7f19a77fe640 1 -- 172.21.15.102:0/3224247864 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a80146d0 con 0x7f19b01348a0 2024-09-13T17:43:56.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.124+0000 7f19b5620640 1 -- 172.21.15.102:0/3224247864 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19b0133680 con 0x7f19b01348a0 2024-09-13T17:43:56.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.125+0000 7f19a77fe640 1 -- 172.21.15.102:0/3224247864 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f19a8003020 con 0x7f19b01348a0 2024-09-13T17:43:56.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.125+0000 7f19a77fe640 1 -- 172.21.15.102:0/3224247864 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f19a8042a20 con 0x7f19b01348a0 2024-09-13T17:43:56.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.125+0000 7f19b5620640 1 -- 172.21.15.102:0/3224247864 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19b00a2880 con 0x7f19b01348a0 2024-09-13T17:43:56.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.130+0000 7f19a77fe640 1 -- 172.21.15.102:0/3224247864 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f19a8018020 con 0x7f19b01348a0 2024-09-13T17:43:56.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.283+0000 7f19b5620640 1 -- 172.21.15.102:0/3224247864 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f19b0064000 con 0x7f19b01348a0 2024-09-13T17:43:56.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.284+0000 7f19a77fe640 1 -- 172.21.15.102:0/3224247864 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f19a80133e0 con 0x7f19b01348a0 2024-09-13T17:43:56.286 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-09-13T17:43:56.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.286+0000 7f19b5620640 1 -- 172.21.15.102:0/3224247864 >> v1:172.21.15.102:6816/34937 conn(0x7f1990031160 legacy=0x7f1990033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.286+0000 7f19b5620640 1 -- 172.21.15.102:0/3224247864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19b01348a0 msgr2=0x7f19b01304a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.286+0000 7f19b5620640 1 --2- 172.21.15.102:0/3224247864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19b01348a0 0x7f19b01304a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f19a8009f40 tx=0x7f19a800b040 comp rx=0 tx=0).stop 2024-09-13T17:43:56.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.286+0000 7f19b5620640 1 -- 172.21.15.102:0/3224247864 shutdown_connections 2024-09-13T17:43:56.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.286+0000 7f19b5620640 1 --2- 172.21.15.102:0/3224247864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19b01348a0 0x7f19b01304a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.286+0000 7f19b5620640 1 --2- 172.21.15.102:0/3224247864 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f19b00698b0 0x7f19b012ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.286+0000 7f19b5620640 1 --2- 172.21.15.102:0/3224247864 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f19b00689d0 0x7f19b012cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.286+0000 7f19b5620640 1 -- 172.21.15.102:0/3224247864 >> 172.21.15.102:0/3224247864 conn(0x7f19b00a2bd0 msgr2=0x7f19b009bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:56.288 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.286+0000 7f19b5620640 1 -- 172.21.15.102:0/3224247864 shutdown_connections 2024-09-13T17:43:56.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.286+0000 7f19b5620640 1 -- 172.21.15.102:0/3224247864 wait complete. 2024-09-13T17:43:56.298 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:43:56.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.390+0000 7f5a5499d640 1 Processor -- start 2024-09-13T17:43:56.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.390+0000 7f5a5499d640 1 -- start start 2024-09-13T17:43:56.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.391+0000 7f5a5499d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5a50128330 0x7f5a50128700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:56.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.391+0000 7f5a5499d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5a5012bf50 0x7f5a50133b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:56.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.391+0000 7f5a5499d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a50097be0 con 0x7f5a50128c40 2024-09-13T17:43:56.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.391+0000 7f5a5499d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a50098e70 con 0x7f5a5012bf50 2024-09-13T17:43:56.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.391+0000 7f5a5499d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a50098310 con 0x7f5a50128330 2024-09-13T17:43:56.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.391+0000 7f5a4ed9d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5a5012bf50 0x7f5a50133b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:56.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.391+0000 7f5a4e59c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5a50128330 0x7f5a50128700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:56.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.391+0000 7f5a4ed9d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f5a5012bf50 0x7f5a50133b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:46404/0 (socket says 172.21.15.102:46404) 2024-09-13T17:43:56.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.391+0000 7f5a4ed9d640 1 -- 172.21.15.102:0/3860901410 learned_addr learned my addr 172.21.15.102:0/3860901410 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:56.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.391+0000 7f5a4dd9b640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f5a50128c40 0x7f5a5012b840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:58938/0 (socket says 172.21.15.102:58938) 2024-09-13T17:43:56.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.391+0000 7f5a4e59c640 1 -- 172.21.15.102:0/3860901410 >> v2:172.21.15.102:3300/0 conn(0x7f5a5012bf50 msgr2=0x7f5a50133b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:56.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.391+0000 7f5a4e59c640 1 --2- 172.21.15.102:0/3860901410 >> v2:172.21.15.102:3300/0 conn(0x7f5a5012bf50 0x7f5a50133b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.391+0000 7f5a4e59c640 1 -- 172.21.15.102:0/3860901410 >> v1:172.21.15.102:6789/0 conn(0x7f5a50128c40 legacy=0x7f5a5012b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.391+0000 7f5a4e59c640 1 -- 172.21.15.102:0/3860901410 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a500985d0 con 0x7f5a50128330 2024-09-13T17:43:56.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.391+0000 7f5a4d59a640 1 -- 172.21.15.102:0/3860901410 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2708546257 0 0) 0x7f5a50097be0 con 0x7f5a50128c40 2024-09-13T17:43:56.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.391+0000 7f5a4ed9d640 1 --2- 172.21.15.102:0/3860901410 >> v2:172.21.15.102:3300/0 conn(0x7f5a5012bf50 0x7f5a50133b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:56.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.392+0000 7f5a4e59c640 1 --2- 172.21.15.102:0/3860901410 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5a50128330 0x7f5a50128700 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000a9f0 tx=0x7f5a4000a2b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79d9a95b1a968c2f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:56.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.392+0000 7f5a4d59a640 1 -- 172.21.15.102:0/3860901410 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a40030510 con 0x7f5a50128330 2024-09-13T17:43:56.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.392+0000 7f5a4d59a640 1 -- 172.21.15.102:0/3860901410 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a4002e050 con 0x7f5a50128330 2024-09-13T17:43:56.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.392+0000 7f5a4d59a640 1 -- 172.21.15.102:0/3860901410 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a400308a0 con 0x7f5a50128330 2024-09-13T17:43:56.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.393+0000 7f5a5499d640 1 -- 172.21.15.102:0/3860901410 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5a50128330 msgr2=0x7f5a50128700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.393+0000 7f5a5499d640 1 --2- 172.21.15.102:0/3860901410 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5a50128330 0x7f5a50128700 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000a9f0 tx=0x7f5a4000a2b0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.393+0000 7f5a5499d640 1 -- 172.21.15.102:0/3860901410 shutdown_connections 2024-09-13T17:43:56.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.393+0000 7f5a5499d640 1 --2- 172.21.15.102:0/3860901410 >> v2:172.21.15.102:3300/0 conn(0x7f5a5012bf50 0x7f5a50133b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.393+0000 7f5a5499d640 1 --2- 172.21.15.102:0/3860901410 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5a50128330 0x7f5a50128700 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.393+0000 7f5a5499d640 1 -- 172.21.15.102:0/3860901410 >> 172.21.15.102:0/3860901410 conn(0x7f5a5009e820 msgr2=0x7f5a5009ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:56.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.393+0000 7f5a5499d640 1 -- 172.21.15.102:0/3860901410 shutdown_connections 2024-09-13T17:43:56.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.393+0000 7f5a5499d640 1 -- 172.21.15.102:0/3860901410 wait complete. 2024-09-13T17:43:56.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.394+0000 7f5a5499d640 1 Processor -- start 2024-09-13T17:43:56.398 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.394+0000 7f5a5499d640 1 -- start start 2024-09-13T17:43:56.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.394+0000 7f5a5499d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5a50128330 0x7f5a5013c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:56.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.394+0000 7f5a5499d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a50128c40 0x7f5a5013ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:56.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.394+0000 7f5a5499d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5a5012bf50 0x7f5a5006b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:56.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.394+0000 7f5a5499d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5009ba60 con 0x7f5a50128c40 2024-09-13T17:43:56.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.394+0000 7f5a5499d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5006e7a0 con 0x7f5a5012bf50 2024-09-13T17:43:56.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.394+0000 7f5a5499d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a50131630 con 0x7f5a50128330 2024-09-13T17:43:56.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.394+0000 7f5a4e59c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5a50128330 0x7f5a5013c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:56.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.394+0000 7f5a4dd9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a50128c40 0x7f5a5013ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:56.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.394+0000 7f5a4ed9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5a5012bf50 0x7f5a5006b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:56.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.394+0000 7f5a4e59c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5a50128330 0x7f5a5013c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:36386/0 (socket says 172.21.15.102:36386) 2024-09-13T17:43:56.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.394+0000 7f5a4e59c640 1 -- 172.21.15.102:0/3290435122 learned_addr learned my addr 172.21.15.102:0/3290435122 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:56.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.394+0000 7f5a4dd9b640 1 -- 172.21.15.102:0/3290435122 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5a50128330 msgr2=0x7f5a5013c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.394+0000 7f5a4dd9b640 1 --2- 172.21.15.102:0/3290435122 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5a50128330 0x7f5a5013c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.394+0000 7f5a4dd9b640 1 -- 172.21.15.102:0/3290435122 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5a5012bf50 msgr2=0x7f5a5006b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.394+0000 7f5a4dd9b640 1 --2- 172.21.15.102:0/3290435122 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5a5012bf50 0x7f5a5006b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.394+0000 7f5a4dd9b640 1 -- 172.21.15.102:0/3290435122 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a48009ae0 con 0x7f5a50128c40 2024-09-13T17:43:56.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.394+0000 7f5a4e59c640 1 --2- 172.21.15.102:0/3290435122 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5a50128330 0x7f5a5013c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:56.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.394+0000 7f5a4ed9d640 1 --2- 172.21.15.102:0/3290435122 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5a5012bf50 0x7f5a5006b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:56.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.395+0000 7f5a4dd9b640 1 --2- 172.21.15.102:0/3290435122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a50128c40 0x7f5a5013ce20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5a44000c00 tx=0x7f5a44007ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:56.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.395+0000 7f5a3e7fc640 1 -- 172.21.15.102:0/3290435122 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a440072e0 con 0x7f5a50128c40 2024-09-13T17:43:56.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.395+0000 7f5a3e7fc640 1 -- 172.21.15.102:0/3290435122 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a44007460 con 0x7f5a50128c40 2024-09-13T17:43:56.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.395+0000 7f5a5499d640 1 -- 172.21.15.102:0/3290435122 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a4000a530 con 0x7f5a50128c40 2024-09-13T17:43:56.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.395+0000 7f5a3e7fc640 1 -- 172.21.15.102:0/3290435122 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a440049f0 con 0x7f5a50128c40 2024-09-13T17:43:56.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.395+0000 7f5a5499d640 1 -- 172.21.15.102:0/3290435122 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a50115390 con 0x7f5a50128c40 2024-09-13T17:43:56.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.396+0000 7f5a3e7fc640 1 -- 172.21.15.102:0/3290435122 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f5a44013400 con 0x7f5a50128c40 2024-09-13T17:43:56.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.396+0000 7f5a3e7fc640 1 -- 172.21.15.102:0/3290435122 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f5a44041420 con 0x7f5a50128c40 2024-09-13T17:43:56.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.396+0000 7f5a5499d640 1 -- 172.21.15.102:0/3290435122 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a50064000 con 0x7f5a50128c40 2024-09-13T17:43:56.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.400+0000 7f5a3e7fc640 1 -- 172.21.15.102:0/3290435122 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f5a440136b0 con 0x7f5a50128c40 2024-09-13T17:43:56.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.584+0000 7f5a5499d640 1 -- 172.21.15.102:0/3290435122 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a50067690 con 0x7f5a50128c40 2024-09-13T17:43:56.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.584+0000 7f5a3e7fc640 1 -- 172.21.15.102:0/3290435122 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2266 (secure 0 0 0) 0x7f5a44010300 con 0x7f5a50128c40 2024-09-13T17:43:56.587 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:43:56.587 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:42:06.611970+0000","modified":"2024-09-13T17:42:09.305093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4210":{"gid":4210,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6817/3443280887","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":3443280887}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-09-13T17:43:56.587 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 5 2024-09-13T17:43:56.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.586+0000 7f5a5499d640 1 -- 172.21.15.102:0/3290435122 >> v1:172.21.15.102:6816/34937 conn(0x7f5a30031160 legacy=0x7f5a30033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.586+0000 7f5a5499d640 1 -- 172.21.15.102:0/3290435122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a50128c40 msgr2=0x7f5a5013ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.586+0000 7f5a5499d640 1 --2- 172.21.15.102:0/3290435122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a50128c40 0x7f5a5013ce20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5a44000c00 tx=0x7f5a44007ca0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.586+0000 7f5a5499d640 1 -- 172.21.15.102:0/3290435122 shutdown_connections 2024-09-13T17:43:56.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.586+0000 7f5a5499d640 1 --2- 172.21.15.102:0/3290435122 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5a5012bf50 0x7f5a5006b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.586+0000 7f5a5499d640 1 --2- 172.21.15.102:0/3290435122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a50128c40 0x7f5a5013ce20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.586+0000 7f5a5499d640 1 --2- 172.21.15.102:0/3290435122 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f5a50128330 0x7f5a5013c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.586+0000 7f5a5499d640 1 -- 172.21.15.102:0/3290435122 >> 172.21.15.102:0/3290435122 conn(0x7f5a5009e820 msgr2=0x7f5a5012c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:56.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.587+0000 7f5a5499d640 1 -- 172.21.15.102:0/3290435122 shutdown_connections 2024-09-13T17:43:56.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.587+0000 7f5a5499d640 1 -- 172.21.15.102:0/3290435122 wait complete. 2024-09-13T17:43:56.599 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:43:56.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.691+0000 7efc4f95d640 1 Processor -- start 2024-09-13T17:43:56.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.691+0000 7efc4f95d640 1 -- start start 2024-09-13T17:43:56.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.691+0000 7efc4f95d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efc4812c870 0x7efc48131490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:56.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.691+0000 7efc4f95d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7efc4812cc80 0x7efc481339e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:56.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.691+0000 7efc4f95d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc4806a530 con 0x7efc4809abb0 2024-09-13T17:43:56.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.691+0000 7efc4f95d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc4809af80 con 0x7efc4812cc80 2024-09-13T17:43:56.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.691+0000 7efc4f95d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efc48098be0 con 0x7efc4812c870 2024-09-13T17:43:56.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.691+0000 7efc4defa640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7efc4812cc80 0x7efc481339e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:56.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.691+0000 7efc4cef8640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efc4812c870 0x7efc48131490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:56.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.691+0000 7efc4d6f9640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7efc4809abb0 0x7efc4812c160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:58946/0 (socket says 172.21.15.102:58946) 2024-09-13T17:43:56.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.691+0000 7efc4defa640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7efc4812cc80 0x7efc481339e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:46420/0 (socket says 172.21.15.102:46420) 2024-09-13T17:43:56.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.691+0000 7efc4d6f9640 1 -- 172.21.15.102:0/438282554 learned_addr learned my addr 172.21.15.102:0/438282554 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:56.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.692+0000 7efc4defa640 1 -- 172.21.15.102:0/438282554 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efc4812c870 msgr2=0x7efc48131490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:56.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.692+0000 7efc4defa640 1 --2- 172.21.15.102:0/438282554 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efc4812c870 0x7efc48131490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.692+0000 7efc4defa640 1 -- 172.21.15.102:0/438282554 >> v1:172.21.15.102:6789/0 conn(0x7efc4809abb0 legacy=0x7efc4812c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.692+0000 7efc3ffff640 1 -- 172.21.15.102:0/438282554 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3673688709 0 0) 0x7efc4806a530 con 0x7efc4809abb0 2024-09-13T17:43:56.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.692+0000 7efc4defa640 1 -- 172.21.15.102:0/438282554 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc48098930 con 0x7efc4812cc80 2024-09-13T17:43:56.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.692+0000 7efc4cef8640 1 --2- 172.21.15.102:0/438282554 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efc4812c870 0x7efc48131490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:56.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.692+0000 7efc4defa640 1 --2- 172.21.15.102:0/438282554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc4812cc80 0x7efc481339e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7efc40009e90 tx=0x7efc40009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c4e312391c7e1d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:56.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.693+0000 7efc3ffff640 1 -- 172.21.15.102:0/438282554 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc40007550 con 0x7efc4812cc80 2024-09-13T17:43:56.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.693+0000 7efc3ffff640 1 -- 172.21.15.102:0/438282554 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4002d050 con 0x7efc4812cc80 2024-09-13T17:43:56.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.693+0000 7efc3ffff640 1 -- 172.21.15.102:0/438282554 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc400078e0 con 0x7efc4812cc80 2024-09-13T17:43:56.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.693+0000 7efc4f95d640 1 -- 172.21.15.102:0/438282554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc4812cc80 msgr2=0x7efc481339e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.693+0000 7efc4f95d640 1 --2- 172.21.15.102:0/438282554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc4812cc80 0x7efc481339e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7efc40009e90 tx=0x7efc40009780 comp rx=0 tx=0).stop 2024-09-13T17:43:56.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.693+0000 7efc4f95d640 1 -- 172.21.15.102:0/438282554 shutdown_connections 2024-09-13T17:43:56.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.693+0000 7efc4f95d640 1 --2- 172.21.15.102:0/438282554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc4812cc80 0x7efc481339e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.693+0000 7efc4f95d640 1 --2- 172.21.15.102:0/438282554 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efc4812c870 0x7efc48131490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.693+0000 7efc4f95d640 1 -- 172.21.15.102:0/438282554 >> 172.21.15.102:0/438282554 conn(0x7efc4809e820 msgr2=0x7efc4809ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:56.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.693+0000 7efc4f95d640 1 -- 172.21.15.102:0/438282554 shutdown_connections 2024-09-13T17:43:56.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.693+0000 7efc4f95d640 1 -- 172.21.15.102:0/438282554 wait complete. 2024-09-13T17:43:56.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.694+0000 7efc4f95d640 1 Processor -- start 2024-09-13T17:43:56.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.694+0000 7efc4f95d640 1 -- start start 2024-09-13T17:43:56.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.695+0000 7efc4f95d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc4809abb0 0x7efc4811baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:56.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.695+0000 7efc4f95d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efc4812c870 0x7efc4811c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:56.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.695+0000 7efc4f95d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efc4812cc80 0x7efc4811c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:56.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.695+0000 7efc4f95d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4811fee0 con 0x7efc4809abb0 2024-09-13T17:43:56.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.695+0000 7efc4f95d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc4811cf60 con 0x7efc4812c870 2024-09-13T17:43:56.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.695+0000 7efc4f95d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efc4811f200 con 0x7efc4812cc80 2024-09-13T17:43:56.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.695+0000 7efc4d6f9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc4809abb0 0x7efc4811baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:56.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.695+0000 7efc4cef8640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efc4812c870 0x7efc4811c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:56.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.695+0000 7efc4d6f9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc4809abb0 0x7efc4811baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:46432/0 (socket says 172.21.15.102:46432) 2024-09-13T17:43:56.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.695+0000 7efc4defa640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efc4812cc80 0x7efc4811c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:56.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.695+0000 7efc4d6f9640 1 -- 172.21.15.102:0/3542500717 learned_addr learned my addr 172.21.15.102:0/3542500717 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:56.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.695+0000 7efc4cef8640 1 -- 172.21.15.102:0/3542500717 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efc4812cc80 msgr2=0x7efc4811c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.695+0000 7efc4cef8640 1 --2- 172.21.15.102:0/3542500717 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efc4812cc80 0x7efc4811c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.695+0000 7efc4cef8640 1 -- 172.21.15.102:0/3542500717 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc4809abb0 msgr2=0x7efc4811baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.695+0000 7efc4cef8640 1 --2- 172.21.15.102:0/3542500717 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc4809abb0 0x7efc4811baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.695+0000 7efc4cef8640 1 -- 172.21.15.102:0/3542500717 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc38009390 con 0x7efc4812c870 2024-09-13T17:43:56.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.695+0000 7efc4d6f9640 1 --2- 172.21.15.102:0/3542500717 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc4809abb0 0x7efc4811baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:56.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.695+0000 7efc4defa640 1 --2- 172.21.15.102:0/3542500717 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efc4812cc80 0x7efc4811c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:56.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.695+0000 7efc4cef8640 1 --2- 172.21.15.102:0/3542500717 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efc4812c870 0x7efc4811c030 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efc3800ea50 tx=0x7efc380090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:56.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.695+0000 7efc3d7fa640 1 -- 172.21.15.102:0/3542500717 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3800ca10 con 0x7efc4812c870 2024-09-13T17:43:56.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.695+0000 7efc3d7fa640 1 -- 172.21.15.102:0/3542500717 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3800cb70 con 0x7efc4812c870 2024-09-13T17:43:56.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.695+0000 7efc4f95d640 1 -- 172.21.15.102:0/3542500717 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc40009a20 con 0x7efc4812c870 2024-09-13T17:43:56.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.695+0000 7efc3d7fa640 1 -- 172.21.15.102:0/3542500717 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc38013470 con 0x7efc4812c870 2024-09-13T17:43:56.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.695+0000 7efc4f95d640 1 -- 172.21.15.102:0/3542500717 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc4811fa60 con 0x7efc4812c870 2024-09-13T17:43:56.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.696+0000 7efc3d7fa640 1 -- 172.21.15.102:0/3542500717 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7efc38013860 con 0x7efc4812c870 2024-09-13T17:43:56.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.696+0000 7efc3d7fa640 1 -- 172.21.15.102:0/3542500717 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7efc380423b0 con 0x7efc4812c870 2024-09-13T17:43:56.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.696+0000 7efc4f95d640 1 -- 172.21.15.102:0/3542500717 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc4811fa60 con 0x7efc4812c870 2024-09-13T17:43:56.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.701+0000 7efc3d7fa640 1 -- 172.21.15.102:0/3542500717 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7efc4811fa60 con 0x7efc4812c870 2024-09-13T17:43:56.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.852+0000 7efc4f95d640 1 -- 172.21.15.102:0/3542500717 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc4811f340 con 0x7efc4812c870 2024-09-13T17:43:56.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.853+0000 7efc3d7fa640 1 -- 172.21.15.102:0/3542500717 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+11660 (secure 0 0 0) 0x7efc4811f340 con 0x7efc4812c870 2024-09-13T17:43:56.855 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:43:56.855 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":23,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:42:48.385896+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:04.672265+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:05.583134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.17","mappings":[{"from":3,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]},{"pgid":"4.25","mappings":[{"from":3,"to":2}]},{"pgid":"4.33","mappings":[{"from":3,"to":0}]},{"pgid":"4.3a","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:43:56.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.855+0000 7efc4f95d640 1 -- 172.21.15.102:0/3542500717 >> v1:172.21.15.102:6816/34937 conn(0x7efc2c031190 legacy=0x7efc2c033640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.855+0000 7efc4f95d640 1 -- 172.21.15.102:0/3542500717 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efc4812c870 msgr2=0x7efc4811c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.855+0000 7efc4f95d640 1 --2- 172.21.15.102:0/3542500717 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efc4812c870 0x7efc4811c030 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efc3800ea50 tx=0x7efc380090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.855+0000 7efc4f95d640 1 -- 172.21.15.102:0/3542500717 shutdown_connections 2024-09-13T17:43:56.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.855+0000 7efc4f95d640 1 --2- 172.21.15.102:0/3542500717 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efc4812cc80 0x7efc4811c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.855+0000 7efc4f95d640 1 --2- 172.21.15.102:0/3542500717 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efc4812c870 0x7efc4811c030 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.855+0000 7efc4f95d640 1 --2- 172.21.15.102:0/3542500717 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc4809abb0 0x7efc4811baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.855+0000 7efc4f95d640 1 -- 172.21.15.102:0/3542500717 >> 172.21.15.102:0/3542500717 conn(0x7efc4809e820 msgr2=0x7efc4809b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:56.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.855+0000 7efc4f95d640 1 -- 172.21.15.102:0/3542500717 shutdown_connections 2024-09-13T17:43:56.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.855+0000 7efc4f95d640 1 -- 172.21.15.102:0/3542500717 wait complete. 2024-09-13T17:43:56.868 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T17:43:56.868 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:43:56.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.959+0000 7f446b8e8640 1 Processor -- start 2024-09-13T17:43:56.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.959+0000 7f446b8e8640 1 -- start start 2024-09-13T17:43:56.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.960+0000 7f446b8e8640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f446409abb0 0x7f446412c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:56.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.960+0000 7f446b8e8640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f446412c6a0 0x7f446412cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:56.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.960+0000 7f446b8e8640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4464096d80 0x7f4464097200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:56.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.960+0000 7f446b8e8640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f446409af80 con 0x7f446409abb0 2024-09-13T17:43:56.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.960+0000 7f446b8e8640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f446412cff0 con 0x7f4464096d80 2024-09-13T17:43:56.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.960+0000 7f446b8e8640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f446406a530 con 0x7f446412c6a0 2024-09-13T17:43:56.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.960+0000 7f4469684640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f446409abb0 0x7f446412c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:56.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.960+0000 7f4468e83640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f446412c6a0 0x7f446412cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:56.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.960+0000 7f4469e85640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4464096d80 0x7f4464097200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:56.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.960+0000 7f4469684640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f446409abb0 0x7f446412c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:46444/0 (socket says 172.21.15.102:46444) 2024-09-13T17:43:56.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.960+0000 7f4469684640 1 -- 172.21.15.102:0/1698377696 learned_addr learned my addr 172.21.15.102:0/1698377696 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:56.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.960+0000 7f4469684640 1 -- 172.21.15.102:0/1698377696 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f446412c6a0 msgr2=0x7f446412cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:56.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.960+0000 7f4469684640 1 --2- 172.21.15.102:0/1698377696 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f446412c6a0 0x7f446412cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.960+0000 7f4469684640 1 -- 172.21.15.102:0/1698377696 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4464096d80 msgr2=0x7f4464097200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:56.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.960+0000 7f4469684640 1 --2- 172.21.15.102:0/1698377696 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4464096d80 0x7f4464097200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.960+0000 7f4469684640 1 -- 172.21.15.102:0/1698377696 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4464098be0 con 0x7f446409abb0 2024-09-13T17:43:56.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.960+0000 7f4469e85640 1 --2- 172.21.15.102:0/1698377696 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4464096d80 0x7f4464097200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:56.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.961+0000 7f4469684640 1 --2- 172.21.15.102:0/1698377696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f446409abb0 0x7f446412c160 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f445c009940 tx=0x7f445c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9394dc13d5e818 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:56.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.961+0000 7f4463fff640 1 -- 172.21.15.102:0/1698377696 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c02ac70 con 0x7f446409abb0 2024-09-13T17:43:56.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.961+0000 7f4463fff640 1 -- 172.21.15.102:0/1698377696 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f445c02add0 con 0x7f446409abb0 2024-09-13T17:43:56.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.961+0000 7f4463fff640 1 -- 172.21.15.102:0/1698377696 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c032670 con 0x7f446409abb0 2024-09-13T17:43:56.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.962+0000 7f446b8e8640 1 -- 172.21.15.102:0/1698377696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f446409abb0 msgr2=0x7f446412c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.962+0000 7f446b8e8640 1 --2- 172.21.15.102:0/1698377696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f446409abb0 0x7f446412c160 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f445c009940 tx=0x7f445c0091d0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.962+0000 7f446b8e8640 1 -- 172.21.15.102:0/1698377696 shutdown_connections 2024-09-13T17:43:56.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.962+0000 7f446b8e8640 1 --2- 172.21.15.102:0/1698377696 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4464096d80 0x7f4464097200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.962+0000 7f446b8e8640 1 --2- 172.21.15.102:0/1698377696 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f446412c6a0 0x7f446412cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.962+0000 7f446b8e8640 1 --2- 172.21.15.102:0/1698377696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f446409abb0 0x7f446412c160 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.962+0000 7f446b8e8640 1 -- 172.21.15.102:0/1698377696 >> 172.21.15.102:0/1698377696 conn(0x7f446409e820 msgr2=0x7f446409ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:56.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.962+0000 7f446b8e8640 1 -- 172.21.15.102:0/1698377696 shutdown_connections 2024-09-13T17:43:56.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.962+0000 7f446b8e8640 1 -- 172.21.15.102:0/1698377696 wait complete. 2024-09-13T17:43:56.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.963+0000 7f446b8e8640 1 Processor -- start 2024-09-13T17:43:56.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.963+0000 7f446b8e8640 1 -- start start 2024-09-13T17:43:56.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.963+0000 7f446b8e8640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4464096d80 0x7f446411ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:56.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.963+0000 7f446b8e8640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f446409abb0 0x7f446411b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:56.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.963+0000 7f446b8e8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f446412c6a0 0x7f446411b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:56.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.963+0000 7f446b8e8640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446411ce80 con 0x7f446412c6a0 2024-09-13T17:43:56.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.963+0000 7f446b8e8640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f446411d8c0 con 0x7f4464096d80 2024-09-13T17:43:56.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.963+0000 7f446b8e8640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f446411c940 con 0x7f446409abb0 2024-09-13T17:43:56.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.963+0000 7f4469684640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4464096d80 0x7f446411ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:56.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.963+0000 7f4468e83640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f446409abb0 0x7f446411b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:56.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.963+0000 7f4469684640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4464096d80 0x7f446411ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:38848/0 (socket says 172.21.15.102:38848) 2024-09-13T17:43:56.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.963+0000 7f4469684640 1 -- 172.21.15.102:0/1896828613 learned_addr learned my addr 172.21.15.102:0/1896828613 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:56.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.963+0000 7f4469e85640 1 --2- 172.21.15.102:0/1896828613 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f446412c6a0 0x7f446411b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:56.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.964+0000 7f4468e83640 1 -- 172.21.15.102:0/1896828613 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4464096d80 msgr2=0x7f446411ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.964+0000 7f4468e83640 1 --2- 172.21.15.102:0/1896828613 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4464096d80 0x7f446411ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.964+0000 7f4468e83640 1 -- 172.21.15.102:0/1896828613 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f446412c6a0 msgr2=0x7f446411b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.964+0000 7f4468e83640 1 --2- 172.21.15.102:0/1896828613 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f446412c6a0 0x7f446411b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.964+0000 7f4468e83640 1 -- 172.21.15.102:0/1896828613 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4454009a20 con 0x7f446409abb0 2024-09-13T17:43:56.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.964+0000 7f4469684640 1 --2- 172.21.15.102:0/1896828613 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4464096d80 0x7f446411ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:56.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.964+0000 7f4469e85640 1 --2- 172.21.15.102:0/1896828613 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f446412c6a0 0x7f446411b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:56.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.964+0000 7f4468e83640 1 --2- 172.21.15.102:0/1896828613 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f446409abb0 0x7f446411b390 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f445800e960 tx=0x7f44580090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:56.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.964+0000 7f44617fa640 1 -- 172.21.15.102:0/1896828613 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445800ee90 con 0x7f446409abb0 2024-09-13T17:43:56.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.964+0000 7f44617fa640 1 -- 172.21.15.102:0/1896828613 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f445800c940 con 0x7f446409abb0 2024-09-13T17:43:56.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.964+0000 7f446b8e8640 1 -- 172.21.15.102:0/1896828613 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f445c009450 con 0x7f446409abb0 2024-09-13T17:43:56.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.964+0000 7f44617fa640 1 -- 172.21.15.102:0/1896828613 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445800cce0 con 0x7f446409abb0 2024-09-13T17:43:56.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.964+0000 7f446b8e8640 1 -- 172.21.15.102:0/1896828613 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f446411ef10 con 0x7f446409abb0 2024-09-13T17:43:56.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.965+0000 7f44617fa640 1 -- 172.21.15.102:0/1896828613 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f4458013680 con 0x7f446409abb0 2024-09-13T17:43:56.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.965+0000 7f44617fa640 1 -- 172.21.15.102:0/1896828613 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f44580422f0 con 0x7f446409abb0 2024-09-13T17:43:56.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.965+0000 7f446b8e8640 1 -- 172.21.15.102:0/1896828613 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4464098e00 con 0x7f446409abb0 2024-09-13T17:43:56.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:56.969+0000 7f44617fa640 1 -- 172.21.15.102:0/1896828613 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f4458021020 con 0x7f446409abb0 2024-09-13T17:43:57.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.152+0000 7f446b8e8640 1 -- 172.21.15.102:0/1896828613 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44640981a0 con 0x7f446409abb0 2024-09-13T17:43:57.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.153+0000 7f44617fa640 1 -- 172.21.15.102:0/1896828613 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2266 (secure 0 0 0) 0x7f44580123d0 con 0x7f446409abb0 2024-09-13T17:43:57.155 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:43:57.155 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:42:06.611970+0000","modified":"2024-09-13T17:42:09.305093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4210":{"gid":4210,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6817/3443280887","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":3443280887}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-09-13T17:43:57.155 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 5 2024-09-13T17:43:57.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.155+0000 7f446b8e8640 1 -- 172.21.15.102:0/1896828613 >> v1:172.21.15.102:6816/34937 conn(0x7f4450031160 legacy=0x7f4450033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:57.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.155+0000 7f446b8e8640 1 -- 172.21.15.102:0/1896828613 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f446409abb0 msgr2=0x7f446411b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:57.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.155+0000 7f446b8e8640 1 --2- 172.21.15.102:0/1896828613 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f446409abb0 0x7f446411b390 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f445800e960 tx=0x7f44580090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:57.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.155+0000 7f446b8e8640 1 -- 172.21.15.102:0/1896828613 shutdown_connections 2024-09-13T17:43:57.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.155+0000 7f446b8e8640 1 --2- 172.21.15.102:0/1896828613 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f446412c6a0 0x7f446411b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:57.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.155+0000 7f446b8e8640 1 --2- 172.21.15.102:0/1896828613 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f446409abb0 0x7f446411b390 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:57.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.155+0000 7f446b8e8640 1 --2- 172.21.15.102:0/1896828613 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4464096d80 0x7f446411ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:57.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.155+0000 7f446b8e8640 1 -- 172.21.15.102:0/1896828613 >> 172.21.15.102:0/1896828613 conn(0x7f446409e820 msgr2=0x7f44641295a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:57.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.155+0000 7f446b8e8640 1 -- 172.21.15.102:0/1896828613 shutdown_connections 2024-09-13T17:43:57.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.155+0000 7f446b8e8640 1 -- 172.21.15.102:0/1896828613 wait complete. 2024-09-13T17:43:57.168 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:43:57.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.258+0000 7face3f9d640 1 Processor -- start 2024-09-13T17:43:57.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.258+0000 7face3f9d640 1 -- start start 2024-09-13T17:43:57.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.259+0000 7face3f9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facdc068920 0x7facdc0a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:57.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.259+0000 7face3f9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7facdc0a1110 0x7facdc0a1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:57.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.259+0000 7face3f9d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7facdc12f540 0x7facdc12f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:57.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.259+0000 7face3f9d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facdc068cf0 con 0x7facdc12f540 2024-09-13T17:43:57.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.259+0000 7face3f9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facdc069850 con 0x7facdc0a1110 2024-09-13T17:43:57.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.259+0000 7face3f9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7facdc0a1a60 con 0x7facdc068920 2024-09-13T17:43:57.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.259+0000 7face1538640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7facdc0a1110 0x7facdc0a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:57.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.259+0000 7face253a640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7facdc12f540 0x7facdc12f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:57.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.259+0000 7face1d39640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facdc068920 0x7facdc0a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:57.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.259+0000 7face1538640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7facdc0a1110 0x7facdc0a1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:38860/0 (socket says 172.21.15.102:38860) 2024-09-13T17:43:57.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.259+0000 7face1538640 1 -- 172.21.15.102:0/777280708 learned_addr learned my addr 172.21.15.102:0/777280708 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:57.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.259+0000 7face1538640 1 -- 172.21.15.102:0/777280708 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facdc068920 msgr2=0x7facdc0a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:57.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.259+0000 7face1538640 1 --2- 172.21.15.102:0/777280708 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facdc068920 0x7facdc0a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:57.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.259+0000 7face1538640 1 -- 172.21.15.102:0/777280708 >> v2:172.21.15.102:3300/0 conn(0x7facdc12f540 msgr2=0x7facdc12f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:57.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.259+0000 7face1538640 1 --2- 172.21.15.102:0/777280708 >> v2:172.21.15.102:3300/0 conn(0x7facdc12f540 0x7facdc12f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:57.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.259+0000 7face1538640 1 -- 172.21.15.102:0/777280708 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facdc069260 con 0x7facdc0a1110 2024-09-13T17:43:57.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.259+0000 7face1d39640 1 --2- 172.21.15.102:0/777280708 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facdc068920 0x7facdc0a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:57.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.259+0000 7face253a640 1 --2- 172.21.15.102:0/777280708 >> v2:172.21.15.102:3300/0 conn(0x7facdc12f540 0x7facdc12f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:57.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.260+0000 7face1538640 1 --2- 172.21.15.102:0/777280708 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7facdc0a1110 0x7facdc0a1520 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7faccc009910 tx=0x7faccc0091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce956c2aad8e682d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:57.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.260+0000 7face0d37640 1 -- 172.21.15.102:0/777280708 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc02ac80 con 0x7facdc0a1110 2024-09-13T17:43:57.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.260+0000 7face0d37640 1 -- 172.21.15.102:0/777280708 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faccc02ade0 con 0x7facdc0a1110 2024-09-13T17:43:57.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.260+0000 7face0d37640 1 -- 172.21.15.102:0/777280708 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc032670 con 0x7facdc0a1110 2024-09-13T17:43:57.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.261+0000 7face3f9d640 1 -- 172.21.15.102:0/777280708 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7facdc0a1110 msgr2=0x7facdc0a1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:57.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.261+0000 7face3f9d640 1 --2- 172.21.15.102:0/777280708 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7facdc0a1110 0x7facdc0a1520 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7faccc009910 tx=0x7faccc0091d0 comp rx=0 tx=0).stop 2024-09-13T17:43:57.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.261+0000 7face3f9d640 1 -- 172.21.15.102:0/777280708 shutdown_connections 2024-09-13T17:43:57.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.261+0000 7face3f9d640 1 --2- 172.21.15.102:0/777280708 >> v2:172.21.15.102:3300/0 conn(0x7facdc12f540 0x7facdc12f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:57.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.261+0000 7face3f9d640 1 --2- 172.21.15.102:0/777280708 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7facdc0a1110 0x7facdc0a1520 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:57.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.261+0000 7face3f9d640 1 --2- 172.21.15.102:0/777280708 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facdc068920 0x7facdc0a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:57.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.261+0000 7face3f9d640 1 -- 172.21.15.102:0/777280708 >> 172.21.15.102:0/777280708 conn(0x7facdc0a3490 msgr2=0x7facdc06e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:57.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.261+0000 7face3f9d640 1 -- 172.21.15.102:0/777280708 shutdown_connections 2024-09-13T17:43:57.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.261+0000 7face3f9d640 1 -- 172.21.15.102:0/777280708 wait complete. 2024-09-13T17:43:57.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.262+0000 7face3f9d640 1 Processor -- start 2024-09-13T17:43:57.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.262+0000 7face3f9d640 1 -- start start 2024-09-13T17:43:57.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.262+0000 7face3f9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facdc068920 0x7facdc13ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:57.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.262+0000 7face3f9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7facdc0a1110 0x7facdc13f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:57.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.262+0000 7face3f9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7facdc12f540 0x7facdc119060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:57.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.262+0000 7face3f9d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facdc11cbe0 con 0x7facdc12f540 2024-09-13T17:43:57.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.262+0000 7face3f9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facdc11bc50 con 0x7facdc0a1110 2024-09-13T17:43:57.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.262+0000 7face3f9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7facdc12ecb0 con 0x7facdc068920 2024-09-13T17:43:57.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.262+0000 7face1d39640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facdc068920 0x7facdc13ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:57.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.262+0000 7face1538640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7facdc0a1110 0x7facdc13f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:57.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.262+0000 7face253a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7facdc12f540 0x7facdc119060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:57.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.262+0000 7face1d39640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facdc068920 0x7facdc13ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:36440/0 (socket says 172.21.15.102:36440) 2024-09-13T17:43:57.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.262+0000 7face1538640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7facdc0a1110 0x7facdc13f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:38864/0 (socket says 172.21.15.102:38864) 2024-09-13T17:43:57.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.262+0000 7face1d39640 1 -- 172.21.15.102:0/2680640103 learned_addr learned my addr 172.21.15.102:0/2680640103 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:57.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.262+0000 7face1d39640 1 -- 172.21.15.102:0/2680640103 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7facdc0a1110 msgr2=0x7facdc13f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:57.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.262+0000 7face1d39640 1 --2- 172.21.15.102:0/2680640103 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7facdc0a1110 0x7facdc13f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:57.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.262+0000 7face1d39640 1 -- 172.21.15.102:0/2680640103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7facdc12f540 msgr2=0x7facdc119060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:57.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.262+0000 7face1d39640 1 --2- 172.21.15.102:0/2680640103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7facdc12f540 0x7facdc119060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:57.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.262+0000 7face1d39640 1 -- 172.21.15.102:0/2680640103 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd8009390 con 0x7facdc068920 2024-09-13T17:43:57.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.262+0000 7face1538640 1 --2- 172.21.15.102:0/2680640103 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7facdc0a1110 0x7facdc13f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:57.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.263+0000 7face1d39640 1 --2- 172.21.15.102:0/2680640103 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facdc068920 0x7facdc13ec20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7facd800ec80 tx=0x7facd80090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:57.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.263+0000 7facd1ffb640 1 -- 172.21.15.102:0/2680640103 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd800eec0 con 0x7facdc068920 2024-09-13T17:43:57.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.263+0000 7facd1ffb640 1 -- 172.21.15.102:0/2680640103 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd800cb00 con 0x7facdc068920 2024-09-13T17:43:57.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.263+0000 7face3f9d640 1 -- 172.21.15.102:0/2680640103 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facd4009a20 con 0x7facdc068920 2024-09-13T17:43:57.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.263+0000 7facd1ffb640 1 -- 172.21.15.102:0/2680640103 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd800ce80 con 0x7facdc068920 2024-09-13T17:43:57.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.263+0000 7face3f9d640 1 -- 172.21.15.102:0/2680640103 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faccc009450 con 0x7facdc068920 2024-09-13T17:43:57.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.264+0000 7facd1ffb640 1 -- 172.21.15.102:0/2680640103 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7facd8013680 con 0x7facdc068920 2024-09-13T17:43:57.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.264+0000 7facd1ffb640 1 -- 172.21.15.102:0/2680640103 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7facd8042490 con 0x7facdc068920 2024-09-13T17:43:57.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.264+0000 7face3f9d640 1 -- 172.21.15.102:0/2680640103 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facdc064000 con 0x7facdc068920 2024-09-13T17:43:57.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.268+0000 7facd1ffb640 1 -- 172.21.15.102:0/2680640103 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7facd8021020 con 0x7facdc068920 2024-09-13T17:43:57.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.452+0000 7face3f9d640 1 -- 172.21.15.102:0/2680640103 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7facdc09c330 con 0x7facdc068920 2024-09-13T17:43:57.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.452+0000 7facd1ffb640 1 -- 172.21.15.102:0/2680640103 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2266 (secure 0 0 0) 0x7facd80236b0 con 0x7facdc068920 2024-09-13T17:43:57.454 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:43:57.454 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:42:06.611970+0000","modified":"2024-09-13T17:42:09.305093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4210":{"gid":4210,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6817/3443280887","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":3443280887}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-09-13T17:43:57.455 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 5 2024-09-13T17:43:57.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.454+0000 7face3f9d640 1 -- 172.21.15.102:0/2680640103 >> v1:172.21.15.102:6816/34937 conn(0x7facc0031160 legacy=0x7facc0033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:57.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.454+0000 7face3f9d640 1 -- 172.21.15.102:0/2680640103 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facdc068920 msgr2=0x7facdc13ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:57.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.454+0000 7face3f9d640 1 --2- 172.21.15.102:0/2680640103 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facdc068920 0x7facdc13ec20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7facd800ec80 tx=0x7facd80090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:57.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.454+0000 7face3f9d640 1 -- 172.21.15.102:0/2680640103 shutdown_connections 2024-09-13T17:43:57.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.454+0000 7face3f9d640 1 --2- 172.21.15.102:0/2680640103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7facdc12f540 0x7facdc119060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:57.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.454+0000 7face3f9d640 1 --2- 172.21.15.102:0/2680640103 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7facdc0a1110 0x7facdc13f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:57.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.454+0000 7face3f9d640 1 --2- 172.21.15.102:0/2680640103 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7facdc068920 0x7facdc13ec20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:57.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.454+0000 7face3f9d640 1 -- 172.21.15.102:0/2680640103 >> 172.21.15.102:0/2680640103 conn(0x7facdc0a3490 msgr2=0x7facdc132650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:57.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.454+0000 7face3f9d640 1 -- 172.21.15.102:0/2680640103 shutdown_connections 2024-09-13T17:43:57.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.454+0000 7face3f9d640 1 -- 172.21.15.102:0/2680640103 wait complete. 2024-09-13T17:43:57.466 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:43:57.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.557+0000 7f63cbb09640 1 Processor -- start 2024-09-13T17:43:57.559 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.557+0000 7f63cbb09640 1 -- start start 2024-09-13T17:43:57.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.558+0000 7f63cbb09640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63c4068920 0x7f63c409d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:57.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.558+0000 7f63cbb09640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f63c409dc50 0x7f63c409e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:57.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.558+0000 7f63cbb09640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f63c412f310 0x7f63c412f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:57.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.558+0000 7f63cbb09640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63c4068cf0 con 0x7f63c409dc50 2024-09-13T17:43:57.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.558+0000 7f63cbb09640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63c40695a0 con 0x7f63c4068920 2024-09-13T17:43:57.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.558+0000 7f63cbb09640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f63c409f090 con 0x7f63c412f310 2024-09-13T17:43:57.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.558+0000 7f63c90a4640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f63c409dc50 0x7f63c409e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:57.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.558+0000 7f63ca0a6640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f63c412f310 0x7f63c412f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:57.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.558+0000 7f63ca0a6640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f63c412f310 0x7f63c412f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:36456/0 (socket says 172.21.15.102:36456) 2024-09-13T17:43:57.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.558+0000 7f63c98a5640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63c4068920 0x7f63c409d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:57.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.558+0000 7f63ca0a6640 1 -- 172.21.15.102:0/96059872 learned_addr learned my addr 172.21.15.102:0/96059872 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:57.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.558+0000 7f63c90a4640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f63c409dc50 0x7f63c409e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:46482/0 (socket says 172.21.15.102:46482) 2024-09-13T17:43:57.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.558+0000 7f63ca0a6640 1 -- 172.21.15.102:0/96059872 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63c4068920 msgr2=0x7f63c409d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:57.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.558+0000 7f63ca0a6640 1 --2- 172.21.15.102:0/96059872 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63c4068920 0x7f63c409d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:57.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.558+0000 7f63ca0a6640 1 -- 172.21.15.102:0/96059872 >> v2:172.21.15.102:3300/0 conn(0x7f63c409dc50 msgr2=0x7f63c409e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:57.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.558+0000 7f63ca0a6640 1 --2- 172.21.15.102:0/96059872 >> v2:172.21.15.102:3300/0 conn(0x7f63c409dc50 0x7f63c409e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:57.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.558+0000 7f63ca0a6640 1 -- 172.21.15.102:0/96059872 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c409cce0 con 0x7f63c412f310 2024-09-13T17:43:57.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.558+0000 7f63c90a4640 1 --2- 172.21.15.102:0/96059872 >> v2:172.21.15.102:3300/0 conn(0x7f63c409dc50 0x7f63c409e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:57.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.558+0000 7f63c98a5640 1 --2- 172.21.15.102:0/96059872 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63c4068920 0x7f63c409d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:57.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.559+0000 7f63ca0a6640 1 --2- 172.21.15.102:0/96059872 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f63c412f310 0x7f63c412f790 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f63b4009f20 tx=0x7f63b4009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1b406605d50b166 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:57.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.559+0000 7f63c88a3640 1 -- 172.21.15.102:0/96059872 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b4031400 con 0x7f63c412f310 2024-09-13T17:43:57.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.559+0000 7f63c88a3640 1 -- 172.21.15.102:0/96059872 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b402d050 con 0x7f63c412f310 2024-09-13T17:43:57.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.559+0000 7f63c88a3640 1 -- 172.21.15.102:0/96059872 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b40317d0 con 0x7f63c412f310 2024-09-13T17:43:57.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.560+0000 7f63cbb09640 1 -- 172.21.15.102:0/96059872 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f63c412f310 msgr2=0x7f63c412f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:57.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.560+0000 7f63cbb09640 1 --2- 172.21.15.102:0/96059872 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f63c412f310 0x7f63c412f790 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f63b4009f20 tx=0x7f63b4009840 comp rx=0 tx=0).stop 2024-09-13T17:43:57.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.560+0000 7f63cbb09640 1 -- 172.21.15.102:0/96059872 shutdown_connections 2024-09-13T17:43:57.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.560+0000 7f63cbb09640 1 --2- 172.21.15.102:0/96059872 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f63c412f310 0x7f63c412f790 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:57.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.560+0000 7f63cbb09640 1 --2- 172.21.15.102:0/96059872 >> v2:172.21.15.102:3300/0 conn(0x7f63c409dc50 0x7f63c409e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:57.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.560+0000 7f63cbb09640 1 --2- 172.21.15.102:0/96059872 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63c4068920 0x7f63c409d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:57.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.560+0000 7f63cbb09640 1 -- 172.21.15.102:0/96059872 >> 172.21.15.102:0/96059872 conn(0x7f63c406f320 msgr2=0x7f63c40a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:57.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.560+0000 7f63cbb09640 1 -- 172.21.15.102:0/96059872 shutdown_connections 2024-09-13T17:43:57.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.560+0000 7f63cbb09640 1 -- 172.21.15.102:0/96059872 wait complete. 2024-09-13T17:43:57.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.561+0000 7f63cbb09640 1 Processor -- start 2024-09-13T17:43:57.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.561+0000 7f63cbb09640 1 -- start start 2024-09-13T17:43:57.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.561+0000 7f63cbb09640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c4068920 0x7f63c411f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:57.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.561+0000 7f63cbb09640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f63c409dc50 0x7f63c411beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:57.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.561+0000 7f63cbb09640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63c412f310 0x7f63c411c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:57.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.561+0000 7f63cbb09640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c409fdb0 con 0x7f63c4068920 2024-09-13T17:43:57.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.561+0000 7f63cbb09640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63c409e620 con 0x7f63c412f310 2024-09-13T17:43:57.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.561+0000 7f63cbb09640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f63c411eaf0 con 0x7f63c409dc50 2024-09-13T17:43:57.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.562+0000 7f63c90a4640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f63c409dc50 0x7f63c411beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:57.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.562+0000 7f63c90a4640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f63c409dc50 0x7f63c411beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:36466/0 (socket says 172.21.15.102:36466) 2024-09-13T17:43:57.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.562+0000 7f63c90a4640 1 -- 172.21.15.102:0/4292694654 learned_addr learned my addr 172.21.15.102:0/4292694654 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:57.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.562+0000 7f63ca0a6640 1 --2- 172.21.15.102:0/4292694654 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63c412f310 0x7f63c411c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:57.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.562+0000 7f63c98a5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c4068920 0x7f63c411f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:57.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.562+0000 7f63c98a5640 1 -- 172.21.15.102:0/4292694654 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f63c409dc50 msgr2=0x7f63c411beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:57.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.562+0000 7f63c98a5640 1 --2- 172.21.15.102:0/4292694654 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f63c409dc50 0x7f63c411beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:57.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.562+0000 7f63c98a5640 1 -- 172.21.15.102:0/4292694654 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63c412f310 msgr2=0x7f63c411c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:57.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.562+0000 7f63c98a5640 1 --2- 172.21.15.102:0/4292694654 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63c412f310 0x7f63c411c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:57.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.562+0000 7f63c98a5640 1 -- 172.21.15.102:0/4292694654 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63bc009390 con 0x7f63c4068920 2024-09-13T17:43:57.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.562+0000 7f63ca0a6640 1 --2- 172.21.15.102:0/4292694654 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63c412f310 0x7f63c411c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:57.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.562+0000 7f63c98a5640 1 --2- 172.21.15.102:0/4292694654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c4068920 0x7f63c411f590 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f63bc00e970 tx=0x7f63bc0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:57.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.562+0000 7f63c1ffb640 1 -- 172.21.15.102:0/4292694654 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc00c940 con 0x7f63c4068920 2024-09-13T17:43:57.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.562+0000 7f63c1ffb640 1 -- 172.21.15.102:0/4292694654 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63bc00caa0 con 0x7f63c4068920 2024-09-13T17:43:57.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.562+0000 7f63c1ffb640 1 -- 172.21.15.102:0/4292694654 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc00ce60 con 0x7f63c4068920 2024-09-13T17:43:57.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.562+0000 7f63cbb09640 1 -- 172.21.15.102:0/4292694654 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63b8009390 con 0x7f63c4068920 2024-09-13T17:43:57.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.563+0000 7f63cbb09640 1 -- 172.21.15.102:0/4292694654 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b4009ae0 con 0x7f63c4068920 2024-09-13T17:43:57.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.563+0000 7f63c1ffb640 1 -- 172.21.15.102:0/4292694654 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f63bc013680 con 0x7f63c4068920 2024-09-13T17:43:57.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.564+0000 7f63c1ffb640 1 -- 172.21.15.102:0/4292694654 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7f63bc042790 con 0x7f63c4068920 2024-09-13T17:43:57.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.564+0000 7f63cbb09640 1 -- 172.21.15.102:0/4292694654 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63c4064000 con 0x7f63c4068920 2024-09-13T17:43:57.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.568+0000 7f63c1ffb640 1 -- 172.21.15.102:0/4292694654 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f63bc021080 con 0x7f63c4068920 2024-09-13T17:43:57.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.721+0000 7f63cbb09640 1 -- 172.21.15.102:0/4292694654 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63c409c7d0 con 0x7f63c4068920 2024-09-13T17:43:57.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.722+0000 7f63c1ffb640 1 -- 172.21.15.102:0/4292694654 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+11660 (secure 0 0 0) 0x7f63bc01b770 con 0x7f63c4068920 2024-09-13T17:43:57.724 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:43:57.724 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":23,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:42:48.385896+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:04.672265+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:05.583134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.17","mappings":[{"from":3,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]},{"pgid":"4.25","mappings":[{"from":3,"to":2}]},{"pgid":"4.33","mappings":[{"from":3,"to":0}]},{"pgid":"4.3a","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:43:57.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.723+0000 7f63cbb09640 1 -- 172.21.15.102:0/4292694654 >> v1:172.21.15.102:6816/34937 conn(0x7f63b0031160 legacy=0x7f63b0033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:57.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.723+0000 7f63cbb09640 1 -- 172.21.15.102:0/4292694654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c4068920 msgr2=0x7f63c411f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:57.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.723+0000 7f63cbb09640 1 --2- 172.21.15.102:0/4292694654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c4068920 0x7f63c411f590 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f63bc00e970 tx=0x7f63bc0090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:57.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.724+0000 7f63cbb09640 1 -- 172.21.15.102:0/4292694654 shutdown_connections 2024-09-13T17:43:57.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.724+0000 7f63cbb09640 1 --2- 172.21.15.102:0/4292694654 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63c412f310 0x7f63c411c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:57.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.724+0000 7f63cbb09640 1 --2- 172.21.15.102:0/4292694654 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f63c409dc50 0x7f63c411beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:57.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.724+0000 7f63cbb09640 1 --2- 172.21.15.102:0/4292694654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63c4068920 0x7f63c411f590 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:57.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.724+0000 7f63cbb09640 1 -- 172.21.15.102:0/4292694654 >> 172.21.15.102:0/4292694654 conn(0x7f63c406f320 msgr2=0x7f63c4098510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:57.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.724+0000 7f63cbb09640 1 -- 172.21.15.102:0/4292694654 shutdown_connections 2024-09-13T17:43:57.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.724+0000 7f63cbb09640 1 -- 172.21.15.102:0/4292694654 wait complete. 2024-09-13T17:43:57.736 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-13T17:43:57.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.828+0000 7fb854a78640 1 Processor -- start 2024-09-13T17:43:57.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.828+0000 7fb854a78640 1 -- start start 2024-09-13T17:43:57.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.829+0000 7fb854a78640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb850128330 0x7fb850128700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:57.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.829+0000 7fb854a78640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb850128c40 0x7fb85012b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:57.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.829+0000 7fb854a78640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb85012bd80 0x7fb85012c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:57.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.829+0000 7fb854a78640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb85012c710 con 0x7fb850128330 2024-09-13T17:43:57.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.829+0000 7fb854a78640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb850097be0 con 0x7fb85012bd80 2024-09-13T17:43:57.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.829+0000 7fb854a78640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb850098e70 con 0x7fb850128c40 2024-09-13T17:43:57.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.829+0000 7fb84dd9b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb850128c40 0x7fb85012b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:57.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.829+0000 7fb84ed9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb85012bd80 0x7fb85012c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:57.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.829+0000 7fb84e59c640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fb850128330 0x7fb850128700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:57.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.829+0000 7fb84dd9b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb850128c40 0x7fb85012b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:36480/0 (socket says 172.21.15.102:36480) 2024-09-13T17:43:57.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.829+0000 7fb84ed9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb85012bd80 0x7fb85012c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:38892/0 (socket says 172.21.15.102:38892) 2024-09-13T17:43:57.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.829+0000 7fb84dd9b640 1 -- 172.21.15.102:0/3091305398 learned_addr learned my addr 172.21.15.102:0/3091305398 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:57.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.829+0000 7fb84ed9d640 1 -- 172.21.15.102:0/3091305398 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb850128c40 msgr2=0x7fb85012b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:57.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.829+0000 7fb84ed9d640 1 --2- 172.21.15.102:0/3091305398 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb850128c40 0x7fb85012b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:57.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.829+0000 7fb84ed9d640 1 -- 172.21.15.102:0/3091305398 >> v2:172.21.15.102:3300/0 conn(0x7fb850128330 msgr2=0x7fb850128700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:57.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.829+0000 7fb84ed9d640 1 --2- 172.21.15.102:0/3091305398 >> v2:172.21.15.102:3300/0 conn(0x7fb850128330 0x7fb850128700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:57.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.829+0000 7fb84ed9d640 1 -- 172.21.15.102:0/3091305398 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb850098310 con 0x7fb85012bd80 2024-09-13T17:43:57.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.829+0000 7fb84e59c640 1 --2- 172.21.15.102:0/3091305398 >> v2:172.21.15.102:3300/0 conn(0x7fb850128330 0x7fb850128700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:57.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.829+0000 7fb84dd9b640 1 --2- 172.21.15.102:0/3091305398 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb850128c40 0x7fb85012b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:57.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.830+0000 7fb84ed9d640 1 --2- 172.21.15.102:0/3091305398 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb85012bd80 0x7fb85012c1d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb84800d0d0 tx=0x7fb84800b990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a57acc96ac32ddef server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:57.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.830+0000 7fb84d59a640 1 -- 172.21.15.102:0/3091305398 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84800b2e0 con 0x7fb85012bd80 2024-09-13T17:43:57.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.830+0000 7fb84d59a640 1 -- 172.21.15.102:0/3091305398 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb848009550 con 0x7fb85012bd80 2024-09-13T17:43:57.834 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.830+0000 7fb84d59a640 1 -- 172.21.15.102:0/3091305398 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb848009930 con 0x7fb85012bd80 2024-09-13T17:43:57.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.831+0000 7fb854a78640 1 -- 172.21.15.102:0/3091305398 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb85012bd80 msgr2=0x7fb85012c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:57.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.831+0000 7fb854a78640 1 --2- 172.21.15.102:0/3091305398 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb85012bd80 0x7fb85012c1d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb84800d0d0 tx=0x7fb84800b990 comp rx=0 tx=0).stop 2024-09-13T17:43:57.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.831+0000 7fb854a78640 1 -- 172.21.15.102:0/3091305398 shutdown_connections 2024-09-13T17:43:57.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.831+0000 7fb854a78640 1 --2- 172.21.15.102:0/3091305398 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb85012bd80 0x7fb85012c1d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:57.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.831+0000 7fb854a78640 1 --2- 172.21.15.102:0/3091305398 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb850128c40 0x7fb85012b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:57.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.831+0000 7fb854a78640 1 --2- 172.21.15.102:0/3091305398 >> v2:172.21.15.102:3300/0 conn(0x7fb850128330 0x7fb850128700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:57.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.831+0000 7fb854a78640 1 -- 172.21.15.102:0/3091305398 >> 172.21.15.102:0/3091305398 conn(0x7fb85009e820 msgr2=0x7fb85009ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:57.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.831+0000 7fb854a78640 1 -- 172.21.15.102:0/3091305398 shutdown_connections 2024-09-13T17:43:57.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.831+0000 7fb854a78640 1 -- 172.21.15.102:0/3091305398 wait complete. 2024-09-13T17:43:57.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.832+0000 7fb854a78640 1 Processor -- start 2024-09-13T17:43:57.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.832+0000 7fb854a78640 1 -- start start 2024-09-13T17:43:57.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.832+0000 7fb854a78640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb850128330 0x7fb850119290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:57.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.832+0000 7fb854a78640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb850128c40 0x7fb8501197d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:57.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.832+0000 7fb854a78640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb85012bd80 0x7fb850119d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:57.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.832+0000 7fb854a78640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85011b330 con 0x7fb850128c40 2024-09-13T17:43:57.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.832+0000 7fb854a78640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb85009a0a0 con 0x7fb85012bd80 2024-09-13T17:43:57.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.832+0000 7fb854a78640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb850125c80 con 0x7fb850128330 2024-09-13T17:43:57.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.832+0000 7fb84e59c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb850128330 0x7fb850119290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:57.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.832+0000 7fb84dd9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb850128c40 0x7fb8501197d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:57.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.832+0000 7fb84e59c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb850128330 0x7fb850119290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:36484/0 (socket says 172.21.15.102:36484) 2024-09-13T17:43:57.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.832+0000 7fb84e59c640 1 -- 172.21.15.102:0/4230109362 learned_addr learned my addr 172.21.15.102:0/4230109362 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:57.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.832+0000 7fb84ed9d640 1 --2- 172.21.15.102:0/4230109362 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb85012bd80 0x7fb850119d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:57.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.832+0000 7fb84dd9b640 1 -- 172.21.15.102:0/4230109362 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb850128330 msgr2=0x7fb850119290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:57.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.832+0000 7fb84dd9b640 1 --2- 172.21.15.102:0/4230109362 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb850128330 0x7fb850119290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:57.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.832+0000 7fb84dd9b640 1 -- 172.21.15.102:0/4230109362 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb85012bd80 msgr2=0x7fb850119d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:57.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.832+0000 7fb84dd9b640 1 --2- 172.21.15.102:0/4230109362 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb85012bd80 0x7fb850119d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:57.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.833+0000 7fb84dd9b640 1 -- 172.21.15.102:0/4230109362 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb840009390 con 0x7fb850128c40 2024-09-13T17:43:57.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.833+0000 7fb84ed9d640 1 --2- 172.21.15.102:0/4230109362 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb85012bd80 0x7fb850119d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:57.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.833+0000 7fb84e59c640 1 --2- 172.21.15.102:0/4230109362 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb850128330 0x7fb850119290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:57.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.833+0000 7fb84dd9b640 1 --2- 172.21.15.102:0/4230109362 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb850128c40 0x7fb8501197d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb84400fb70 tx=0x7fb84400a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:57.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.833+0000 7fb83e7fc640 1 -- 172.21.15.102:0/4230109362 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb844009040 con 0x7fb850128c40 2024-09-13T17:43:57.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.833+0000 7fb83e7fc640 1 -- 172.21.15.102:0/4230109362 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb84400a7e0 con 0x7fb850128c40 2024-09-13T17:43:57.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.833+0000 7fb854a78640 1 -- 172.21.15.102:0/4230109362 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb84400a500 con 0x7fb850128c40 2024-09-13T17:43:57.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.833+0000 7fb83e7fc640 1 -- 172.21.15.102:0/4230109362 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84400de10 con 0x7fb850128c40 2024-09-13T17:43:57.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.833+0000 7fb854a78640 1 -- 172.21.15.102:0/4230109362 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb84800bc30 con 0x7fb850128c40 2024-09-13T17:43:57.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.834+0000 7fb83e7fc640 1 -- 172.21.15.102:0/4230109362 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fb84401a030 con 0x7fb850128c40 2024-09-13T17:43:57.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.834+0000 7fb83e7fc640 1 -- 172.21.15.102:0/4230109362 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3863+0+0 (secure 0 0 0) 0x7fb844018070 con 0x7fb850128c40 2024-09-13T17:43:57.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.834+0000 7fb854a78640 1 -- 172.21.15.102:0/4230109362 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb850064000 con 0x7fb850128c40 2024-09-13T17:43:57.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:57.838+0000 7fb83e7fc640 1 -- 172.21.15.102:0/4230109362 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fb84401ad50 con 0x7fb850128c40 2024-09-13T17:43:58.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.021+0000 7fb854a78640 1 -- 172.21.15.102:0/4230109362 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb85011a3c0 con 0x7fb850128c40 2024-09-13T17:43:58.023 INFO:tasks.ceph.mon.a.smithi102.stderr:2024-09-13T17:43:58.021+0000 7fa5873f1640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T17:43:58.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.026+0000 7fb83e7fc640 1 -- 172.21.15.102:0/4230109362 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v6) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb84401a750 con 0x7fb850128c40 2024-09-13T17:43:58.028 INFO:teuthology.orchestra.run.smithi102.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-13T17:43:58.029 INFO:tasks.ceph.mds.a.smithi102.stderr: -3132> 2024-09-13T17:42:12.276+0000 7fc94bbf1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:43:58.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.028+0000 7fb854a78640 1 -- 172.21.15.102:0/4230109362 >> v1:172.21.15.102:6816/34937 conn(0x7fb830031160 legacy=0x7fb830033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:58.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.028+0000 7fb854a78640 1 -- 172.21.15.102:0/4230109362 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb850128c40 msgr2=0x7fb8501197d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:58.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.028+0000 7fb854a78640 1 --2- 172.21.15.102:0/4230109362 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb850128c40 0x7fb8501197d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb84400fb70 tx=0x7fb84400a2a0 comp rx=0 tx=0).stop 2024-09-13T17:43:58.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.028+0000 7fb854a78640 1 -- 172.21.15.102:0/4230109362 shutdown_connections 2024-09-13T17:43:58.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.028+0000 7fb854a78640 1 --2- 172.21.15.102:0/4230109362 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fb85012bd80 0x7fb850119d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:58.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.028+0000 7fb854a78640 1 --2- 172.21.15.102:0/4230109362 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb850128c40 0x7fb8501197d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:58.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.028+0000 7fb854a78640 1 --2- 172.21.15.102:0/4230109362 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fb850128330 0x7fb850119290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:58.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.028+0000 7fb854a78640 1 -- 172.21.15.102:0/4230109362 >> 172.21.15.102:0/4230109362 conn(0x7fb85009e820 msgr2=0x7fb850129740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:58.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.028+0000 7fb854a78640 1 -- 172.21.15.102:0/4230109362 shutdown_connections 2024-09-13T17:43:58.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.029+0000 7fb854a78640 1 -- 172.21.15.102:0/4230109362 wait complete. 2024-09-13T17:43:58.042 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-13T17:43:58.053 INFO:tasks.ceph.mds.a.smithi102.stdout:starting mds.a at 2024-09-13T17:43:58.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.134+0000 7f81a4b9d640 1 Processor -- start 2024-09-13T17:43:58.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.134+0000 7f81a4b9d640 1 -- start start 2024-09-13T17:43:58.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.135+0000 7f81a4b9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81a00689d0 0x7f81a009d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:58.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.135+0000 7f81a4b9d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f81a009e160 0x7f81a012fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:58.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.135+0000 7f81a4b9d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81a0096b10 con 0x7f81a009dd70 2024-09-13T17:43:58.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.135+0000 7f81a4b9d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81a0096db0 con 0x7f81a009e160 2024-09-13T17:43:58.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.135+0000 7f81a4b9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81a0097730 con 0x7f81a00689d0 2024-09-13T17:43:58.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.135+0000 7f819e59c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81a00689d0 0x7f81a009d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:58.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.135+0000 7f819ed9d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f81a009e160 0x7f81a012fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:58.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.135+0000 7f819dd9b640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f81a009dd70 0x7f81a012f610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:58990/0 (socket says 172.21.15.102:58990) 2024-09-13T17:43:58.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.135+0000 7f819dd9b640 1 -- 172.21.15.102:0/4196171191 learned_addr learned my addr 172.21.15.102:0/4196171191 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:43:58.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.135+0000 7f819ed9d640 1 -- 172.21.15.102:0/4196171191 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81a00689d0 msgr2=0x7f81a009d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:58.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.135+0000 7f819ed9d640 1 --2- 172.21.15.102:0/4196171191 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81a00689d0 0x7f81a009d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:58.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.135+0000 7f819ed9d640 1 -- 172.21.15.102:0/4196171191 >> v1:172.21.15.102:6789/0 conn(0x7f81a009dd70 legacy=0x7f81a012f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:58.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.135+0000 7f819d59a640 1 -- 172.21.15.102:0/4196171191 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2390297074 0 0) 0x7f81a0096b10 con 0x7f81a009dd70 2024-09-13T17:43:58.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.135+0000 7f819ed9d640 1 -- 172.21.15.102:0/4196171191 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a00696e0 con 0x7f81a009e160 2024-09-13T17:43:58.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.135+0000 7f819e59c640 1 --2- 172.21.15.102:0/4196171191 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81a00689d0 0x7f81a009d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:58.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.136+0000 7f819ed9d640 1 --2- 172.21.15.102:0/4196171191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81a009e160 0x7f81a012fd20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8190009850 tx=0x7f81900090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a10a11c0b414913 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:58.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.136+0000 7f819d59a640 1 -- 172.21.15.102:0/4196171191 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819002ac80 con 0x7f81a009e160 2024-09-13T17:43:58.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.136+0000 7f819d59a640 1 -- 172.21.15.102:0/4196171191 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f819002ade0 con 0x7f81a009e160 2024-09-13T17:43:58.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.136+0000 7f819d59a640 1 -- 172.21.15.102:0/4196171191 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8190032670 con 0x7f81a009e160 2024-09-13T17:43:58.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.137+0000 7f81a4b9d640 1 -- 172.21.15.102:0/4196171191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81a009e160 msgr2=0x7f81a012fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:58.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.137+0000 7f81a4b9d640 1 --2- 172.21.15.102:0/4196171191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81a009e160 0x7f81a012fd20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8190009850 tx=0x7f81900090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:58.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.137+0000 7f81a4b9d640 1 -- 172.21.15.102:0/4196171191 shutdown_connections 2024-09-13T17:43:58.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.137+0000 7f81a4b9d640 1 --2- 172.21.15.102:0/4196171191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81a009e160 0x7f81a012fd20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:58.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.137+0000 7f81a4b9d640 1 --2- 172.21.15.102:0/4196171191 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81a00689d0 0x7f81a009d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:58.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.137+0000 7f81a4b9d640 1 -- 172.21.15.102:0/4196171191 >> 172.21.15.102:0/4196171191 conn(0x7f81a006f320 msgr2=0x7f81a00a2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:58.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.137+0000 7f81a4b9d640 1 -- 172.21.15.102:0/4196171191 shutdown_connections 2024-09-13T17:43:58.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.137+0000 7f81a4b9d640 1 -- 172.21.15.102:0/4196171191 wait complete. 2024-09-13T17:43:58.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.138+0000 7f81a4b9d640 1 Processor -- start 2024-09-13T17:43:58.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.138+0000 7f81a4b9d640 1 -- start start 2024-09-13T17:43:58.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.138+0000 7f81a4b9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f81a00689d0 0x7f81a011e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:58.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.138+0000 7f81a4b9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81a009dd70 0x7f81a011ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:58.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.138+0000 7f81a4b9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81a009e160 0x7f81a011f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:58.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.138+0000 7f81a4b9d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a01205d0 con 0x7f81a009dd70 2024-09-13T17:43:58.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.138+0000 7f81a4b9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a0121ce0 con 0x7f81a00689d0 2024-09-13T17:43:58.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.138+0000 7f81a4b9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81a0122ba0 con 0x7f81a009e160 2024-09-13T17:43:58.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.138+0000 7f819dd9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81a009dd70 0x7f81a011ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:58.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.138+0000 7f819e59c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f81a00689d0 0x7f81a011e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:58.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.138+0000 7f819ed9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81a009e160 0x7f81a011f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:58.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.138+0000 7f819ed9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81a009e160 0x7f81a011f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:36494/0 (socket says 172.21.15.102:36494) 2024-09-13T17:43:58.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.138+0000 7f819e59c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f81a00689d0 0x7f81a011e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:38912/0 (socket says 172.21.15.102:38912) 2024-09-13T17:43:58.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.138+0000 7f819dd9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81a009dd70 0x7f81a011ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:46546/0 (socket says 172.21.15.102:46546) 2024-09-13T17:43:58.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.138+0000 7f819ed9d640 1 -- 172.21.15.102:0/1158138297 learned_addr learned my addr 172.21.15.102:0/1158138297 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:58.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.138+0000 7f819ed9d640 1 -- 172.21.15.102:0/1158138297 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f81a00689d0 msgr2=0x7f81a011e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:58.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.138+0000 7f819ed9d640 1 --2- 172.21.15.102:0/1158138297 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f81a00689d0 0x7f81a011e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:58.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.138+0000 7f819ed9d640 1 -- 172.21.15.102:0/1158138297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81a009dd70 msgr2=0x7f81a011ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:58.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.138+0000 7f819ed9d640 1 --2- 172.21.15.102:0/1158138297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81a009dd70 0x7f81a011ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:58.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.138+0000 7f819ed9d640 1 -- 172.21.15.102:0/1158138297 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8198009a20 con 0x7f81a009e160 2024-09-13T17:43:58.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.138+0000 7f819dd9b640 1 --2- 172.21.15.102:0/1158138297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81a009dd70 0x7f81a011ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:58.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.139+0000 7f819ed9d640 1 --2- 172.21.15.102:0/1158138297 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81a009e160 0x7f81a011f230 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8190035040 tx=0x7f8190032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:58.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.139+0000 7f81967fc640 1 -- 172.21.15.102:0/1158138297 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8190032e40 con 0x7f81a009e160 2024-09-13T17:43:58.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.139+0000 7f81967fc640 1 -- 172.21.15.102:0/1158138297 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8190031a60 con 0x7f81a009e160 2024-09-13T17:43:58.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.139+0000 7f81967fc640 1 -- 172.21.15.102:0/1158138297 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8190031e20 con 0x7f81a009e160 2024-09-13T17:43:58.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.139+0000 7f81a4b9d640 1 -- 172.21.15.102:0/1158138297 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8190009390 con 0x7f81a009e160 2024-09-13T17:43:58.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.139+0000 7f81a4b9d640 1 -- 172.21.15.102:0/1158138297 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a01200b0 con 0x7f81a009e160 2024-09-13T17:43:58.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.139+0000 7f81967fc640 1 -- 172.21.15.102:0/1158138297 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f819003c400 con 0x7f81a009e160 2024-09-13T17:43:58.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.140+0000 7f81967fc640 1 -- 172.21.15.102:0/1158138297 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3906+0+0 (secure 0 0 0) 0x7f81900620f0 con 0x7f81a009e160 2024-09-13T17:43:58.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.140+0000 7f81a4b9d640 1 -- 172.21.15.102:0/1158138297 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81a0064000 con 0x7f81a009e160 2024-09-13T17:43:58.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.144+0000 7f81967fc640 1 -- 172.21.15.102:0/1158138297 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f819003b3a0 con 0x7f81a009e160 2024-09-13T17:43:58.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:58.332+0000 7f81a4b9d640 1 -- 172.21.15.102:0/1158138297 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f81a00a39c0 con 0x7f81a009e160 2024-09-13T17:43:59.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.029+0000 7f81967fc640 1 -- 172.21.15.102:0/1158138297 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7f81900370a0 con 0x7f81a009e160 2024-09-13T17:43:59.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.031+0000 7f81a4b9d640 1 -- 172.21.15.102:0/1158138297 >> v1:172.21.15.102:6816/34937 conn(0x7f8180031160 legacy=0x7f8180033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:59.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.031+0000 7f81a4b9d640 1 -- 172.21.15.102:0/1158138297 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81a009e160 msgr2=0x7f81a011f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:59.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.031+0000 7f81a4b9d640 1 --2- 172.21.15.102:0/1158138297 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81a009e160 0x7f81a011f230 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8190035040 tx=0x7f8190032000 comp rx=0 tx=0).stop 2024-09-13T17:43:59.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.031+0000 7f81a4b9d640 1 -- 172.21.15.102:0/1158138297 shutdown_connections 2024-09-13T17:43:59.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.031+0000 7f81a4b9d640 1 --2- 172.21.15.102:0/1158138297 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81a009e160 0x7f81a011f230 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:59.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.031+0000 7f81a4b9d640 1 --2- 172.21.15.102:0/1158138297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81a009dd70 0x7f81a011ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:59.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.031+0000 7f81a4b9d640 1 --2- 172.21.15.102:0/1158138297 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f81a00689d0 0x7f81a011e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:59.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.031+0000 7f81a4b9d640 1 -- 172.21.15.102:0/1158138297 >> 172.21.15.102:0/1158138297 conn(0x7f81a006f320 msgr2=0x7f81a00a18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:59.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.031+0000 7f81a4b9d640 1 -- 172.21.15.102:0/1158138297 shutdown_connections 2024-09-13T17:43:59.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.031+0000 7f81a4b9d640 1 -- 172.21.15.102:0/1158138297 wait complete. 2024-09-13T17:43:59.045 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-13T17:43:59.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.181+0000 7f570f09f640 1 Processor -- start 2024-09-13T17:43:59.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.181+0000 7f570f09f640 1 -- start start 2024-09-13T17:43:59.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.182+0000 7f570f09f640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f570809a620 0x7f570812d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:59.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.182+0000 7f570f09f640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f570812d910 0x7f570812dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:59.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.182+0000 7f570f09f640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5708096a70 0x7f5708096f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:59.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.182+0000 7f570f09f640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f570809a9f0 con 0x7f570812d910 2024-09-13T17:43:59.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.182+0000 7f570f09f640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f570806b2e0 con 0x7f5708096a70 2024-09-13T17:43:59.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.182+0000 7f570f09f640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f570806a000 con 0x7f570809a620 2024-09-13T17:43:59.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.182+0000 7f5707fff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f570812d910 0x7f570812dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:59.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.182+0000 7f570ce3b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f570809a620 0x7f570812d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:59.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.182+0000 7f570d63c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5708096a70 0x7f5708096f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:59.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.182+0000 7f570ce3b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f570809a620 0x7f570812d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:36508/0 (socket says 172.21.15.102:36508) 2024-09-13T17:43:59.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.182+0000 7f5707fff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f570812d910 0x7f570812dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:46550/0 (socket says 172.21.15.102:46550) 2024-09-13T17:43:59.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.182+0000 7f570d63c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5708096a70 0x7f5708096f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:38920/0 (socket says 172.21.15.102:38920) 2024-09-13T17:43:59.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.182+0000 7f570ce3b640 1 -- 172.21.15.102:0/2115054435 learned_addr learned my addr 172.21.15.102:0/2115054435 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:59.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.182+0000 7f5707fff640 1 -- 172.21.15.102:0/2115054435 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f570809a620 msgr2=0x7f570812d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:59.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.182+0000 7f5707fff640 1 --2- 172.21.15.102:0/2115054435 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f570809a620 0x7f570812d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:59.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.182+0000 7f5707fff640 1 -- 172.21.15.102:0/2115054435 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5708096a70 msgr2=0x7f5708096f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:43:59.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.182+0000 7f5707fff640 1 --2- 172.21.15.102:0/2115054435 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5708096a70 0x7f5708096f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:59.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.182+0000 7f5707fff640 1 -- 172.21.15.102:0/2115054435 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f570806a3c0 con 0x7f570812d910 2024-09-13T17:43:59.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.182+0000 7f570ce3b640 1 --2- 172.21.15.102:0/2115054435 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f570809a620 0x7f570812d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:59.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.182+0000 7f570d63c640 1 --2- 172.21.15.102:0/2115054435 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5708096a70 0x7f5708096f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:43:59.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.183+0000 7f5707fff640 1 --2- 172.21.15.102:0/2115054435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f570812d910 0x7f570812dce0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f56fc009850 tx=0x7f56fc0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d408d5cdd07b64 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:59.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.183+0000 7f57077fe640 1 -- 172.21.15.102:0/2115054435 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56fc02abc0 con 0x7f570812d910 2024-09-13T17:43:59.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.183+0000 7f57077fe640 1 -- 172.21.15.102:0/2115054435 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56fc02ad20 con 0x7f570812d910 2024-09-13T17:43:59.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.183+0000 7f57077fe640 1 -- 172.21.15.102:0/2115054435 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56fc032670 con 0x7f570812d910 2024-09-13T17:43:59.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.183+0000 7f570f09f640 1 -- 172.21.15.102:0/2115054435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f570812d910 msgr2=0x7f570812dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:59.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.183+0000 7f570f09f640 1 --2- 172.21.15.102:0/2115054435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f570812d910 0x7f570812dce0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f56fc009850 tx=0x7f56fc0090f0 comp rx=0 tx=0).stop 2024-09-13T17:43:59.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.183+0000 7f570f09f640 1 -- 172.21.15.102:0/2115054435 shutdown_connections 2024-09-13T17:43:59.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.183+0000 7f570f09f640 1 --2- 172.21.15.102:0/2115054435 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5708096a70 0x7f5708096f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:59.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.183+0000 7f570f09f640 1 --2- 172.21.15.102:0/2115054435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f570812d910 0x7f570812dce0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:59.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.183+0000 7f570f09f640 1 --2- 172.21.15.102:0/2115054435 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f570809a620 0x7f570812d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:59.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.183+0000 7f570f09f640 1 -- 172.21.15.102:0/2115054435 >> 172.21.15.102:0/2115054435 conn(0x7f570809e720 msgr2=0x7f570809eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:59.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.183+0000 7f570f09f640 1 -- 172.21.15.102:0/2115054435 shutdown_connections 2024-09-13T17:43:59.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.184+0000 7f570f09f640 1 -- 172.21.15.102:0/2115054435 wait complete. 2024-09-13T17:43:59.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.184+0000 7f570f09f640 1 Processor -- start 2024-09-13T17:43:59.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.184+0000 7f570f09f640 1 -- start start 2024-09-13T17:43:59.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.185+0000 7f570f09f640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5708096a70 0x7f5708119af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:59.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.185+0000 7f570f09f640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f570809a620 0x7f570811a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:59.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.185+0000 7f570f09f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f570812d910 0x7f570811a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:59.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.185+0000 7f570f09f640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570811de10 con 0x7f570812d910 2024-09-13T17:43:59.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.185+0000 7f570f09f640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f570812b1f0 con 0x7f5708096a70 2024-09-13T17:43:59.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.185+0000 7f570f09f640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f570811af60 con 0x7f570809a620 2024-09-13T17:43:59.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.185+0000 7f570ce3b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5708096a70 0x7f5708119af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:59.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.185+0000 7f5707fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f570809a620 0x7f570811a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:59.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.185+0000 7f570d63c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f570812d910 0x7f570811a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:59.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.185+0000 7f570ce3b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5708096a70 0x7f5708119af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:38934/0 (socket says 172.21.15.102:38934) 2024-09-13T17:43:59.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.185+0000 7f570ce3b640 1 -- 172.21.15.102:0/4048122845 learned_addr learned my addr 172.21.15.102:0/4048122845 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:43:59.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.185+0000 7f570ce3b640 1 -- 172.21.15.102:0/4048122845 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f570809a620 msgr2=0x7f570811a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:59.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.185+0000 7f570ce3b640 1 --2- 172.21.15.102:0/4048122845 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f570809a620 0x7f570811a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:59.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.185+0000 7f570ce3b640 1 -- 172.21.15.102:0/4048122845 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f570812d910 msgr2=0x7f570811a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:59.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.185+0000 7f570ce3b640 1 --2- 172.21.15.102:0/4048122845 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f570812d910 0x7f570811a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:59.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.185+0000 7f570ce3b640 1 -- 172.21.15.102:0/4048122845 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5700009a20 con 0x7f5708096a70 2024-09-13T17:43:59.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.185+0000 7f570ce3b640 1 --2- 172.21.15.102:0/4048122845 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5708096a70 0x7f5708119af0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f56f800ec80 tx=0x7f56f80091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:59.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.185+0000 7f5704ff9640 1 -- 172.21.15.102:0/4048122845 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f800eec0 con 0x7f5708096a70 2024-09-13T17:43:59.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.186+0000 7f5704ff9640 1 -- 172.21.15.102:0/4048122845 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f800cb00 con 0x7f5708096a70 2024-09-13T17:43:59.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.186+0000 7f570f09f640 1 -- 172.21.15.102:0/4048122845 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f8009450 con 0x7f5708096a70 2024-09-13T17:43:59.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.186+0000 7f5704ff9640 1 -- 172.21.15.102:0/4048122845 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f800ce80 con 0x7f5708096a70 2024-09-13T17:43:59.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.186+0000 7f570f09f640 1 -- 172.21.15.102:0/4048122845 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56fc009390 con 0x7f5708096a70 2024-09-13T17:43:59.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.186+0000 7f5704ff9640 1 -- 172.21.15.102:0/4048122845 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f56f8013680 con 0x7f5708096a70 2024-09-13T17:43:59.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.186+0000 7f5704ff9640 1 -- 172.21.15.102:0/4048122845 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3906+0+0 (secure 0 0 0) 0x7f56f8042880 con 0x7f5708096a70 2024-09-13T17:43:59.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.186+0000 7f570f09f640 1 -- 172.21.15.102:0/4048122845 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5708064000 con 0x7f5708096a70 2024-09-13T17:43:59.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.191+0000 7f5704ff9640 1 -- 172.21.15.102:0/4048122845 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f56f8021020 con 0x7f5708096a70 2024-09-13T17:43:59.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:43:59.347+0000 7f570f09f640 1 -- 172.21.15.102:0/4048122845 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f57080981f0 con 0x7f5708096a70 2024-09-13T17:44:00.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.032+0000 7f5704ff9640 1 -- 172.21.15.102:0/4048122845 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) v1 ==== 181+0+0 (secure 0 0 0) 0x7f56f801b590 con 0x7f5708096a70 2024-09-13T17:44:00.034 INFO:teuthology.orchestra.run.smithi102.stderr:pool 'cephfs_metadata' removed 2024-09-13T17:44:00.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.036+0000 7f570f09f640 1 -- 172.21.15.102:0/4048122845 >> v1:172.21.15.102:6816/34937 conn(0x7f56f4031160 legacy=0x7f56f4033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:00.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.036+0000 7f570f09f640 1 -- 172.21.15.102:0/4048122845 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5708096a70 msgr2=0x7f5708119af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:00.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.036+0000 7f570f09f640 1 --2- 172.21.15.102:0/4048122845 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5708096a70 0x7f5708119af0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f56f800ec80 tx=0x7f56f80091d0 comp rx=0 tx=0).stop 2024-09-13T17:44:00.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.036+0000 7f570f09f640 1 -- 172.21.15.102:0/4048122845 shutdown_connections 2024-09-13T17:44:00.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.036+0000 7f570f09f640 1 --2- 172.21.15.102:0/4048122845 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f570812d910 0x7f570811a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:00.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.036+0000 7f570f09f640 1 --2- 172.21.15.102:0/4048122845 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f570809a620 0x7f570811a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:00.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.036+0000 7f570f09f640 1 --2- 172.21.15.102:0/4048122845 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f5708096a70 0x7f5708119af0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:00.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.036+0000 7f570f09f640 1 -- 172.21.15.102:0/4048122845 >> 172.21.15.102:0/4048122845 conn(0x7f570809e720 msgr2=0x7f570806d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:00.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.036+0000 7f570f09f640 1 -- 172.21.15.102:0/4048122845 shutdown_connections 2024-09-13T17:44:00.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.036+0000 7f570f09f640 1 -- 172.21.15.102:0/4048122845 wait complete. 2024-09-13T17:44:00.053 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-13T17:44:00.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.152+0000 7f8eda95d640 1 Processor -- start 2024-09-13T17:44:00.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.152+0000 7f8eda95d640 1 -- start start 2024-09-13T17:44:00.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.152+0000 7f8eda95d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ed4098ed0 0x7f8ed41314a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:00.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.152+0000 7f8eda95d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8ed41319e0 0x7f8ed4133dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:00.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.152+0000 7f8eda95d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ed406a1e0 con 0x7f8ed4096a70 2024-09-13T17:44:00.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.152+0000 7f8eda95d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ed409a900 con 0x7f8ed41319e0 2024-09-13T17:44:00.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.152+0000 7f8eda95d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed4099e80 con 0x7f8ed4098ed0 2024-09-13T17:44:00.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.152+0000 7f8ed8efa640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f8ed41319e0 0x7f8ed4133dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:00.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.152+0000 7f8ed37fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ed4098ed0 0x7f8ed41314a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:00.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.152+0000 7f8ed3fff640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f8ed4096a70 0x7f8ed40987c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:58994/0 (socket says 172.21.15.102:58994) 2024-09-13T17:44:00.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.152+0000 7f8ed3fff640 1 -- 172.21.15.102:0/1549772337 learned_addr learned my addr 172.21.15.102:0/1549772337 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:44:00.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.153+0000 7f8ed37fe640 1 -- 172.21.15.102:0/1549772337 >> v2:172.21.15.102:3300/0 conn(0x7f8ed41319e0 msgr2=0x7f8ed4133dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:00.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.153+0000 7f8ed37fe640 1 --2- 172.21.15.102:0/1549772337 >> v2:172.21.15.102:3300/0 conn(0x7f8ed41319e0 0x7f8ed4133dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:00.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.153+0000 7f8ed37fe640 1 -- 172.21.15.102:0/1549772337 >> v1:172.21.15.102:6789/0 conn(0x7f8ed4096a70 legacy=0x7f8ed40987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:00.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.153+0000 7f8ed37fe640 1 -- 172.21.15.102:0/1549772337 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed406eb50 con 0x7f8ed4098ed0 2024-09-13T17:44:00.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.153+0000 7f8ed8efa640 1 --2- 172.21.15.102:0/1549772337 >> v2:172.21.15.102:3300/0 conn(0x7f8ed41319e0 0x7f8ed4133dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:00.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.153+0000 7f8ed37fe640 1 --2- 172.21.15.102:0/1549772337 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ed4098ed0 0x7f8ed41314a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009850 tx=0x7f8ec80090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f56e68f9f1a61427 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:00.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.153+0000 7f8ed2ffd640 1 -- 172.21.15.102:0/1549772337 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec802ad00 con 0x7f8ed4098ed0 2024-09-13T17:44:00.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.153+0000 7f8ed2ffd640 1 -- 172.21.15.102:0/1549772337 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec802ae60 con 0x7f8ed4098ed0 2024-09-13T17:44:00.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.154+0000 7f8ed2ffd640 1 -- 172.21.15.102:0/1549772337 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8032670 con 0x7f8ed4098ed0 2024-09-13T17:44:00.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.154+0000 7f8eda95d640 1 -- 172.21.15.102:0/1549772337 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ed4098ed0 msgr2=0x7f8ed41314a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:00.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.154+0000 7f8eda95d640 1 --2- 172.21.15.102:0/1549772337 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ed4098ed0 0x7f8ed41314a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009850 tx=0x7f8ec80090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:00.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.154+0000 7f8eda95d640 1 -- 172.21.15.102:0/1549772337 shutdown_connections 2024-09-13T17:44:00.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.154+0000 7f8eda95d640 1 --2- 172.21.15.102:0/1549772337 >> v2:172.21.15.102:3300/0 conn(0x7f8ed41319e0 0x7f8ed4133dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:00.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.154+0000 7f8eda95d640 1 --2- 172.21.15.102:0/1549772337 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ed4098ed0 0x7f8ed41314a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:00.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.154+0000 7f8eda95d640 1 -- 172.21.15.102:0/1549772337 >> 172.21.15.102:0/1549772337 conn(0x7f8ed409e720 msgr2=0x7f8ed409eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:00.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.154+0000 7f8eda95d640 1 -- 172.21.15.102:0/1549772337 shutdown_connections 2024-09-13T17:44:00.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.154+0000 7f8eda95d640 1 -- 172.21.15.102:0/1549772337 wait complete. 2024-09-13T17:44:00.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.155+0000 7f8eda95d640 1 Processor -- start 2024-09-13T17:44:00.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.155+0000 7f8eda95d640 1 -- start start 2024-09-13T17:44:00.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.155+0000 7f8eda95d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8ed4096a70 0x7f8ed411b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:00.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.155+0000 7f8eda95d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ed4098ed0 0x7f8ed411b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:00.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.155+0000 7f8eda95d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ed41319e0 0x7f8ed411bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:00.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.155+0000 7f8eda95d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed4097ce0 con 0x7f8ed41319e0 2024-09-13T17:44:00.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.155+0000 7f8eda95d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed4121c10 con 0x7f8ed4096a70 2024-09-13T17:44:00.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.155+0000 7f8eda95d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed4121ed0 con 0x7f8ed4098ed0 2024-09-13T17:44:00.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.156+0000 7f8ed3fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8ed4096a70 0x7f8ed411b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:00.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.156+0000 7f8ed37fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ed4098ed0 0x7f8ed411b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:00.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.156+0000 7f8ed8efa640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ed41319e0 0x7f8ed411bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:00.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.156+0000 7f8ed3fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8ed4096a70 0x7f8ed411b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:38938/0 (socket says 172.21.15.102:38938) 2024-09-13T17:44:00.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.156+0000 7f8ed3fff640 1 -- 172.21.15.102:0/1634275112 learned_addr learned my addr 172.21.15.102:0/1634275112 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:00.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.156+0000 7f8ed8efa640 1 -- 172.21.15.102:0/1634275112 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ed4098ed0 msgr2=0x7f8ed411b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:00.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.156+0000 7f8ed8efa640 1 --2- 172.21.15.102:0/1634275112 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ed4098ed0 0x7f8ed411b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:00.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.156+0000 7f8ed8efa640 1 -- 172.21.15.102:0/1634275112 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8ed4096a70 msgr2=0x7f8ed411b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:00.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.156+0000 7f8ed8efa640 1 --2- 172.21.15.102:0/1634275112 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8ed4096a70 0x7f8ed411b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:00.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.156+0000 7f8ed8efa640 1 -- 172.21.15.102:0/1634275112 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ecc009a20 con 0x7f8ed41319e0 2024-09-13T17:44:00.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.156+0000 7f8ed37fe640 1 --2- 172.21.15.102:0/1634275112 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ed4098ed0 0x7f8ed411b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:00.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.156+0000 7f8ed3fff640 1 --2- 172.21.15.102:0/1634275112 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8ed4096a70 0x7f8ed411b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:00.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.156+0000 7f8ed8efa640 1 --2- 172.21.15.102:0/1634275112 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ed41319e0 0x7f8ed411bf20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc009ee0 tx=0x7f8ecc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:00.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.156+0000 7f8ebbfff640 1 -- 172.21.15.102:0/1634275112 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc00ce10 con 0x7f8ed41319e0 2024-09-13T17:44:00.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.156+0000 7f8ebbfff640 1 -- 172.21.15.102:0/1634275112 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ecc00f040 con 0x7f8ed41319e0 2024-09-13T17:44:00.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.156+0000 7f8eda95d640 1 -- 172.21.15.102:0/1634275112 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ec8009390 con 0x7f8ed41319e0 2024-09-13T17:44:00.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.156+0000 7f8ebbfff640 1 -- 172.21.15.102:0/1634275112 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc0146d0 con 0x7f8ed41319e0 2024-09-13T17:44:00.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.156+0000 7f8eda95d640 1 -- 172.21.15.102:0/1634275112 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed411c650 con 0x7f8ed41319e0 2024-09-13T17:44:00.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.157+0000 7f8ebbfff640 1 -- 172.21.15.102:0/1634275112 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f8ecc003020 con 0x7f8ed41319e0 2024-09-13T17:44:00.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.157+0000 7f8ebbfff640 1 -- 172.21.15.102:0/1634275112 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3468+0+0 (secure 0 0 0) 0x7f8ecc042930 con 0x7f8ed41319e0 2024-09-13T17:44:00.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.157+0000 7f8eda95d640 1 -- 172.21.15.102:0/1634275112 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ed40976d0 con 0x7f8ed41319e0 2024-09-13T17:44:00.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.161+0000 7f8ebbfff640 1 -- 172.21.15.102:0/1634275112 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f8ecc018020 con 0x7f8ed41319e0 2024-09-13T17:44:00.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:00.312+0000 7f8eda95d640 1 -- 172.21.15.102:0/1634275112 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8ed411c8d0 con 0x7f8ed41319e0 2024-09-13T17:44:01.040 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.038+0000 7f8ebbfff640 1 -- 172.21.15.102:0/1634275112 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8ecc012b30 con 0x7f8ed41319e0 2024-09-13T17:44:01.040 INFO:teuthology.orchestra.run.smithi102.stderr:pool 'cephfs_data' removed 2024-09-13T17:44:01.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.041+0000 7f8eda95d640 1 -- 172.21.15.102:0/1634275112 >> v1:172.21.15.102:6816/34937 conn(0x7f8ec0031160 legacy=0x7f8ec0033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:01.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.041+0000 7f8eda95d640 1 -- 172.21.15.102:0/1634275112 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ed41319e0 msgr2=0x7f8ed411bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:01.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.041+0000 7f8eda95d640 1 --2- 172.21.15.102:0/1634275112 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ed41319e0 0x7f8ed411bf20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc009ee0 tx=0x7f8ecc00b040 comp rx=0 tx=0).stop 2024-09-13T17:44:01.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.041+0000 7f8eda95d640 1 -- 172.21.15.102:0/1634275112 shutdown_connections 2024-09-13T17:44:01.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.041+0000 7f8eda95d640 1 --2- 172.21.15.102:0/1634275112 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ed41319e0 0x7f8ed411bf20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:01.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.041+0000 7f8eda95d640 1 --2- 172.21.15.102:0/1634275112 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8ed4098ed0 0x7f8ed411b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:01.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.041+0000 7f8eda95d640 1 --2- 172.21.15.102:0/1634275112 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8ed4096a70 0x7f8ed411b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:01.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.041+0000 7f8eda95d640 1 -- 172.21.15.102:0/1634275112 >> 172.21.15.102:0/1634275112 conn(0x7f8ed409e720 msgr2=0x7f8ed409d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:01.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.042+0000 7f8eda95d640 1 -- 172.21.15.102:0/1634275112 shutdown_connections 2024-09-13T17:44:01.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.042+0000 7f8eda95d640 1 -- 172.21.15.102:0/1634275112 wait complete. 2024-09-13T17:44:01.058 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T17:44:01.058 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-13T17:44:01.058 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T17:44:01.058 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:44:01.067 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T17:44:01.068 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T17:44:01.110 INFO:tasks.ceph.mds.a:Started 2024-09-13T17:44:01.110 DEBUG:teuthology.parallel:result is None 2024-09-13T17:44:01.111 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-13T17:44:01.180 INFO:tasks.ceph.mds.a.smithi102.stdout:starting mds.a at 2024-09-13T17:44:01.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.204+0000 7f61cf95d640 1 Processor -- start 2024-09-13T17:44:01.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.205+0000 7f61cf95d640 1 -- start start 2024-09-13T17:44:01.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.205+0000 7f61cf95d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f61c8068920 0x7f61c809d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:01.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.205+0000 7f61cf95d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f61c809e020 0x7f61c812fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:01.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.205+0000 7f61cf95d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c80695a0 con 0x7f61c809dc50 2024-09-13T17:44:01.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.205+0000 7f61cf95d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c8068cf0 con 0x7f61c809e020 2024-09-13T17:44:01.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.205+0000 7f61cf95d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f61c809e4a0 con 0x7f61c8068920 2024-09-13T17:44:01.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.205+0000 7f61cd6f9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f61c8068920 0x7f61c809d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:01.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.205+0000 7f61cdefa640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f61c809e020 0x7f61c812fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:01.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.205+0000 7f61ccef8640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f61c809dc50 0x7f61c812f310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:59026/0 (socket says 172.21.15.102:59026) 2024-09-13T17:44:01.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.205+0000 7f61ccef8640 1 -- 172.21.15.102:0/1338776469 learned_addr learned my addr 172.21.15.102:0/1338776469 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:44:01.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.206+0000 7f61cd6f9640 1 -- 172.21.15.102:0/1338776469 >> v2:172.21.15.102:3300/0 conn(0x7f61c809e020 msgr2=0x7f61c812fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:01.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.206+0000 7f61cd6f9640 1 --2- 172.21.15.102:0/1338776469 >> v2:172.21.15.102:3300/0 conn(0x7f61c809e020 0x7f61c812fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:01.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.206+0000 7f61cd6f9640 1 -- 172.21.15.102:0/1338776469 >> v1:172.21.15.102:6789/0 conn(0x7f61c809dc50 legacy=0x7f61c812f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:01.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.206+0000 7f61cd6f9640 1 -- 172.21.15.102:0/1338776469 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c809f090 con 0x7f61c8068920 2024-09-13T17:44:01.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.206+0000 7f61cdefa640 1 --2- 172.21.15.102:0/1338776469 >> v2:172.21.15.102:3300/0 conn(0x7f61c809e020 0x7f61c812fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:01.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.206+0000 7f61cd6f9640 1 --2- 172.21.15.102:0/1338776469 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f61c8068920 0x7f61c809d710 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f61c4009850 tx=0x7f61c40090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18653c075afb8b58 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:01.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.206+0000 7f61bbfff640 1 -- 172.21.15.102:0/1338776469 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c402ac80 con 0x7f61c8068920 2024-09-13T17:44:01.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.206+0000 7f61bbfff640 1 -- 172.21.15.102:0/1338776469 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c402ade0 con 0x7f61c8068920 2024-09-13T17:44:01.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.206+0000 7f61bbfff640 1 -- 172.21.15.102:0/1338776469 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c4032670 con 0x7f61c8068920 2024-09-13T17:44:01.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.207+0000 7f61cf95d640 1 -- 172.21.15.102:0/1338776469 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f61c8068920 msgr2=0x7f61c809d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:01.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.207+0000 7f61cf95d640 1 --2- 172.21.15.102:0/1338776469 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f61c8068920 0x7f61c809d710 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f61c4009850 tx=0x7f61c40090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:01.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.207+0000 7f61cf95d640 1 -- 172.21.15.102:0/1338776469 shutdown_connections 2024-09-13T17:44:01.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.207+0000 7f61cf95d640 1 --2- 172.21.15.102:0/1338776469 >> v2:172.21.15.102:3300/0 conn(0x7f61c809e020 0x7f61c812fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:01.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.207+0000 7f61cf95d640 1 --2- 172.21.15.102:0/1338776469 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f61c8068920 0x7f61c809d710 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:01.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.207+0000 7f61cf95d640 1 -- 172.21.15.102:0/1338776469 >> 172.21.15.102:0/1338776469 conn(0x7f61c806f320 msgr2=0x7f61c80a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:01.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.207+0000 7f61cf95d640 1 -- 172.21.15.102:0/1338776469 shutdown_connections 2024-09-13T17:44:01.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.207+0000 7f61cf95d640 1 -- 172.21.15.102:0/1338776469 wait complete. 2024-09-13T17:44:01.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.208+0000 7f61cf95d640 1 Processor -- start 2024-09-13T17:44:01.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.208+0000 7f61cf95d640 1 -- start start 2024-09-13T17:44:01.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.208+0000 7f61cf95d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f61c8068920 0x7f61c8121bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:01.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.208+0000 7f61cf95d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f61c809dc50 0x7f61c811dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:01.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.208+0000 7f61cf95d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61c809e020 0x7f61c811e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:01.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.208+0000 7f61cf95d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c809e620 con 0x7f61c809e020 2024-09-13T17:44:01.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.208+0000 7f61cf95d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c8121110 con 0x7f61c809dc50 2024-09-13T17:44:01.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.208+0000 7f61cf95d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f61c81216e0 con 0x7f61c8068920 2024-09-13T17:44:01.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.209+0000 7f61cd6f9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f61c8068920 0x7f61c8121bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:01.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.209+0000 7f61cdefa640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61c809e020 0x7f61c811e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:01.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.209+0000 7f61ccef8640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f61c809dc50 0x7f61c811dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:01.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.209+0000 7f61cdefa640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61c809e020 0x7f61c811e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:46612/0 (socket says 172.21.15.102:46612) 2024-09-13T17:44:01.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.209+0000 7f61cdefa640 1 -- 172.21.15.102:0/3591112321 learned_addr learned my addr 172.21.15.102:0/3591112321 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:01.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.209+0000 7f61cdefa640 1 -- 172.21.15.102:0/3591112321 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f61c8068920 msgr2=0x7f61c8121bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:01.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.209+0000 7f61cdefa640 1 --2- 172.21.15.102:0/3591112321 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f61c8068920 0x7f61c8121bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:01.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.209+0000 7f61cdefa640 1 -- 172.21.15.102:0/3591112321 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f61c809dc50 msgr2=0x7f61c811dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:01.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.209+0000 7f61cdefa640 1 --2- 172.21.15.102:0/3591112321 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f61c809dc50 0x7f61c811dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:01.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.209+0000 7f61cdefa640 1 -- 172.21.15.102:0/3591112321 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c0009a20 con 0x7f61c809e020 2024-09-13T17:44:01.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.209+0000 7f61cd6f9640 1 --2- 172.21.15.102:0/3591112321 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f61c8068920 0x7f61c8121bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:01.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.209+0000 7f61cdefa640 1 --2- 172.21.15.102:0/3591112321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61c809e020 0x7f61c811e1d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f61c0009fc0 tx=0x7f61c000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:01.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.209+0000 7f61b97fa640 1 -- 172.21.15.102:0/3591112321 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c000ce70 con 0x7f61c809e020 2024-09-13T17:44:01.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.209+0000 7f61b97fa640 1 -- 172.21.15.102:0/3591112321 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c000f040 con 0x7f61c809e020 2024-09-13T17:44:01.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.209+0000 7f61cf95d640 1 -- 172.21.15.102:0/3591112321 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c4009390 con 0x7f61c809e020 2024-09-13T17:44:01.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.209+0000 7f61b97fa640 1 -- 172.21.15.102:0/3591112321 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c00146d0 con 0x7f61c809e020 2024-09-13T17:44:01.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.209+0000 7f61cf95d640 1 -- 172.21.15.102:0/3591112321 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c811f8b0 con 0x7f61c809e020 2024-09-13T17:44:01.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.210+0000 7f61b97fa640 1 -- 172.21.15.102:0/3591112321 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f61c0003020 con 0x7f61c809e020 2024-09-13T17:44:01.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.210+0000 7f61cf95d640 1 -- 172.21.15.102:0/3591112321 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61c8121330 con 0x7f61c809e020 2024-09-13T17:44:01.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.210+0000 7f61b97fa640 1 -- 172.21.15.102:0/3591112321 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 2851+0+0 (secure 0 0 0) 0x7f61c0042770 con 0x7f61c809e020 2024-09-13T17:44:01.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.215+0000 7f61b97fa640 1 -- 172.21.15.102:0/3591112321 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f61c00133e0 con 0x7f61c809e020 2024-09-13T17:44:01.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:01.394+0000 7f61cf95d640 1 -- 172.21.15.102:0/3591112321 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f61c8064000 con 0x7f61c809e020 2024-09-13T17:44:03.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.039+0000 7f61b97fa640 1 -- 172.21.15.102:0/3591112321 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v28) v1 ==== 97+0+0 (secure 0 0 0) 0x7f61c0018020 con 0x7f61c809e020 2024-09-13T17:44:03.041 INFO:teuthology.orchestra.run.smithi102.stderr: removed all blocklist entries 2024-09-13T17:44:03.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.042+0000 7f61cf95d640 1 -- 172.21.15.102:0/3591112321 >> v1:172.21.15.102:6816/34937 conn(0x7f61ac031190 legacy=0x7f61ac033640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:03.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.042+0000 7f61cf95d640 1 -- 172.21.15.102:0/3591112321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61c809e020 msgr2=0x7f61c811e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:03.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.042+0000 7f61cf95d640 1 --2- 172.21.15.102:0/3591112321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61c809e020 0x7f61c811e1d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f61c0009fc0 tx=0x7f61c000b040 comp rx=0 tx=0).stop 2024-09-13T17:44:03.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.042+0000 7f61cf95d640 1 -- 172.21.15.102:0/3591112321 shutdown_connections 2024-09-13T17:44:03.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.042+0000 7f61cf95d640 1 --2- 172.21.15.102:0/3591112321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61c809e020 0x7f61c811e1d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:03.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.042+0000 7f61cf95d640 1 --2- 172.21.15.102:0/3591112321 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f61c809dc50 0x7f61c811dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:03.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.042+0000 7f61cf95d640 1 --2- 172.21.15.102:0/3591112321 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f61c8068920 0x7f61c8121bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:03.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.042+0000 7f61cf95d640 1 -- 172.21.15.102:0/3591112321 >> 172.21.15.102:0/3591112321 conn(0x7f61c806f320 msgr2=0x7f61c809b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:03.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.042+0000 7f61cf95d640 1 -- 172.21.15.102:0/3591112321 shutdown_connections 2024-09-13T17:44:03.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.042+0000 7f61cf95d640 1 -- 172.21.15.102:0/3591112321 wait complete. 2024-09-13T17:44:03.057 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-13T17:44:03.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.151+0000 7f2ed299d640 1 Processor -- start 2024-09-13T17:44:03.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.152+0000 7f2ed299d640 1 -- start start 2024-09-13T17:44:03.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.152+0000 7f2ed299d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ecc09de20 0x7f2ecc134680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:03.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.152+0000 7f2ed299d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2ecc09e1f0 0x7f2ecc136bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:03.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.152+0000 7f2ed299d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ecc0695a0 con 0x7f2ecc068920 2024-09-13T17:44:03.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.152+0000 7f2ed299d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ecc068cf0 con 0x7f2ecc09e1f0 2024-09-13T17:44:03.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.152+0000 7f2ed299d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc09f090 con 0x7f2ecc09de20 2024-09-13T17:44:03.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.152+0000 7f2ecbfff640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f2ecc068920 0x7f2ecc09d710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:37394/0 (socket says 172.21.15.102:37394) 2024-09-13T17:44:03.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.152+0000 7f2ed0f3a640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2ecc09e1f0 0x7f2ecc136bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:03.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.152+0000 7f2ecbfff640 1 -- 172.21.15.102:0/749449735 learned_addr learned my addr 172.21.15.102:0/749449735 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:44:03.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.152+0000 7f2ec3fff640 1 --2- 172.21.15.102:0/749449735 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ecc09de20 0x7f2ecc134680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:03.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.152+0000 7f2ec3fff640 1 -- 172.21.15.102:0/749449735 >> v2:172.21.15.102:3300/0 conn(0x7f2ecc09e1f0 msgr2=0x7f2ecc136bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:03.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.152+0000 7f2ec3fff640 1 --2- 172.21.15.102:0/749449735 >> v2:172.21.15.102:3300/0 conn(0x7f2ecc09e1f0 0x7f2ecc136bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:03.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.152+0000 7f2ec3fff640 1 -- 172.21.15.102:0/749449735 >> v1:172.21.15.102:6789/0 conn(0x7f2ecc068920 legacy=0x7f2ecc09d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:03.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.152+0000 7f2ecb7fe640 1 -- 172.21.15.102:0/749449735 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1137297358 0 0) 0x7f2ecc0695a0 con 0x7f2ecc068920 2024-09-13T17:44:03.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.152+0000 7f2ec3fff640 1 -- 172.21.15.102:0/749449735 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ecc09cce0 con 0x7f2ecc09de20 2024-09-13T17:44:03.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.152+0000 7f2ed0f3a640 1 --2- 172.21.15.102:0/749449735 >> v2:172.21.15.102:3300/0 conn(0x7f2ecc09e1f0 0x7f2ecc136bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:03.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.153+0000 7f2ec3fff640 1 --2- 172.21.15.102:0/749449735 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ecc09de20 0x7f2ecc134680 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc096e50 tx=0x7f2eb40090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=663532d04462d135 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:03.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.153+0000 7f2ecb7fe640 1 -- 172.21.15.102:0/749449735 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb402ac50 con 0x7f2ecc09de20 2024-09-13T17:44:03.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.153+0000 7f2ecb7fe640 1 -- 172.21.15.102:0/749449735 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb402adb0 con 0x7f2ecc09de20 2024-09-13T17:44:03.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.153+0000 7f2ecb7fe640 1 -- 172.21.15.102:0/749449735 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4032670 con 0x7f2ecc09de20 2024-09-13T17:44:03.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.154+0000 7f2ed299d640 1 -- 172.21.15.102:0/749449735 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ecc09de20 msgr2=0x7f2ecc134680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:03.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.154+0000 7f2ed299d640 1 --2- 172.21.15.102:0/749449735 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ecc09de20 0x7f2ecc134680 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc096e50 tx=0x7f2eb40090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:03.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.154+0000 7f2ed299d640 1 -- 172.21.15.102:0/749449735 shutdown_connections 2024-09-13T17:44:03.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.154+0000 7f2ed299d640 1 --2- 172.21.15.102:0/749449735 >> v2:172.21.15.102:3300/0 conn(0x7f2ecc09e1f0 0x7f2ecc136bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:03.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.154+0000 7f2ed299d640 1 --2- 172.21.15.102:0/749449735 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ecc09de20 0x7f2ecc134680 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:03.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.154+0000 7f2ed299d640 1 -- 172.21.15.102:0/749449735 >> 172.21.15.102:0/749449735 conn(0x7f2ecc06f320 msgr2=0x7f2ecc0a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:03.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.154+0000 7f2ed299d640 1 -- 172.21.15.102:0/749449735 shutdown_connections 2024-09-13T17:44:03.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.154+0000 7f2ed299d640 1 -- 172.21.15.102:0/749449735 wait complete. 2024-09-13T17:44:03.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.155+0000 7f2ed299d640 1 Processor -- start 2024-09-13T17:44:03.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.155+0000 7f2ed299d640 1 -- start start 2024-09-13T17:44:03.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.155+0000 7f2ed299d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ecc068920 0x7f2ecc1340e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:03.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.155+0000 7f2ed299d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ecc09de20 0x7f2ecc134620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:03.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.155+0000 7f2ed299d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ecc09e1f0 0x7f2ecc134b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:03.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.155+0000 7f2ed299d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc11a770 con 0x7f2ecc09de20 2024-09-13T17:44:03.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.155+0000 7f2ed299d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc135cb0 con 0x7f2ecc09e1f0 2024-09-13T17:44:03.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.155+0000 7f2ed299d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc119290 con 0x7f2ecc068920 2024-09-13T17:44:03.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.155+0000 7f2ec3fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ecc09de20 0x7f2ecc134620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:03.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.155+0000 7f2ec3fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ecc09de20 0x7f2ecc134620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34222/0 (socket says 172.21.15.102:34222) 2024-09-13T17:44:03.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.155+0000 7f2ecbfff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ecc068920 0x7f2ecc1340e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:03.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.155+0000 7f2ec3fff640 1 -- 172.21.15.102:0/2362934609 learned_addr learned my addr 172.21.15.102:0/2362934609 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:03.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.155+0000 7f2ed0f3a640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ecc09e1f0 0x7f2ecc134b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:03.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.156+0000 7f2ec3fff640 1 -- 172.21.15.102:0/2362934609 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ecc068920 msgr2=0x7f2ecc1340e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:03.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.156+0000 7f2ec3fff640 1 --2- 172.21.15.102:0/2362934609 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ecc068920 0x7f2ecc1340e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:03.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.156+0000 7f2ec3fff640 1 -- 172.21.15.102:0/2362934609 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ecc09e1f0 msgr2=0x7f2ecc134b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:03.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.156+0000 7f2ec3fff640 1 --2- 172.21.15.102:0/2362934609 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ecc09e1f0 0x7f2ecc134b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:03.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.156+0000 7f2ec3fff640 1 -- 172.21.15.102:0/2362934609 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec4009a20 con 0x7f2ecc09de20 2024-09-13T17:44:03.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.156+0000 7f2ec3fff640 1 --2- 172.21.15.102:0/2362934609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ecc09de20 0x7f2ecc134620 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009880 tx=0x7f2eb4032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:03.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.156+0000 7f2ec8ff9640 1 -- 172.21.15.102:0/2362934609 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4031700 con 0x7f2ecc09de20 2024-09-13T17:44:03.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.156+0000 7f2ec8ff9640 1 -- 172.21.15.102:0/2362934609 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb4031860 con 0x7f2ecc09de20 2024-09-13T17:44:03.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.156+0000 7f2ec8ff9640 1 -- 172.21.15.102:0/2362934609 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4031c20 con 0x7f2ecc09de20 2024-09-13T17:44:03.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.156+0000 7f2ed299d640 1 -- 172.21.15.102:0/2362934609 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb4009390 con 0x7f2ecc09de20 2024-09-13T17:44:03.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.156+0000 7f2ed299d640 1 -- 172.21.15.102:0/2362934609 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ecc09e620 con 0x7f2ecc09de20 2024-09-13T17:44:03.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.157+0000 7f2ec8ff9640 1 -- 172.21.15.102:0/2362934609 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f2eb403c610 con 0x7f2ecc09de20 2024-09-13T17:44:03.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.157+0000 7f2ec8ff9640 1 -- 172.21.15.102:0/2362934609 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 2808+0+0 (secure 0 0 0) 0x7f2eb40623d0 con 0x7f2ecc09de20 2024-09-13T17:44:03.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.157+0000 7f2ed299d640 1 -- 172.21.15.102:0/2362934609 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ecc064000 con 0x7f2ecc09de20 2024-09-13T17:44:03.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.161+0000 7f2ec8ff9640 1 -- 172.21.15.102:0/2362934609 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f2eb4049080 con 0x7f2ecc09de20 2024-09-13T17:44:03.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.369+0000 7f2ed299d640 1 -- 172.21.15.102:0/2362934609 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2ecc11ac00 con 0x7f2ecc09de20 2024-09-13T17:44:03.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.370+0000 7f2ec8ff9640 1 -- 172.21.15.102:0/2362934609 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+3573 (secure 0 0 0) 0x7f2eb4037030 con 0x7f2ecc09de20 2024-09-13T17:44:03.372 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:44:03.372 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-13T17:44:03.372 INFO:teuthology.orchestra.run.smithi102.stdout: "auth_dump": [ 2024-09-13T17:44:03.372 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-13T17:44:03.372 INFO:teuthology.orchestra.run.smithi102.stdout: "entity": "mds.a", 2024-09-13T17:44:03.373 INFO:teuthology.orchestra.run.smithi102.stdout: "key": "AQBAeeRm+mivJxAAV6KWk8hYJYWOfdjhXMjw+A==", 2024-09-13T17:44:03.373 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": { 2024-09-13T17:44:03.373 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": "allow", 2024-09-13T17:44:03.373 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": "allow *", 2024-09-13T17:44:03.373 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": "allow *", 2024-09-13T17:44:03.373 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": "allow *" 2024-09-13T17:44:03.373 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-13T17:44:03.373 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-13T17:44:03.373 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-13T17:44:03.373 INFO:teuthology.orchestra.run.smithi102.stdout: "entity": "osd.0", 2024-09-13T17:44:03.373 INFO:teuthology.orchestra.run.smithi102.stdout: "key": "AQBEeeRmE/BzARAAilYkvotOaS6Fo5QC2J33yQ==", 2024-09-13T17:44:03.373 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": { 2024-09-13T17:44:03.373 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": "allow profile osd", 2024-09-13T17:44:03.374 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": "allow profile osd", 2024-09-13T17:44:03.374 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": "allow *" 2024-09-13T17:44:03.374 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-13T17:44:03.374 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-13T17:44:03.374 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-13T17:44:03.374 INFO:teuthology.orchestra.run.smithi102.stdout: "entity": "osd.1", 2024-09-13T17:44:03.374 INFO:teuthology.orchestra.run.smithi102.stdout: "key": "AQBGeeRm3hZIOxAAE2ewM4Z40LxLSuqvF/+aow==", 2024-09-13T17:44:03.374 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": { 2024-09-13T17:44:03.374 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": "allow profile osd", 2024-09-13T17:44:03.374 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": "allow profile osd", 2024-09-13T17:44:03.374 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": "allow *" 2024-09-13T17:44:03.374 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-13T17:44:03.374 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-13T17:44:03.374 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-13T17:44:03.375 INFO:teuthology.orchestra.run.smithi102.stdout: "entity": "osd.2", 2024-09-13T17:44:03.375 INFO:teuthology.orchestra.run.smithi102.stdout: "key": "AQBJeeRmc+cIOhAAbSGkIDTFg7HyaVJTs0alxA==", 2024-09-13T17:44:03.375 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": { 2024-09-13T17:44:03.375 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": "allow profile osd", 2024-09-13T17:44:03.375 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": "allow profile osd", 2024-09-13T17:44:03.375 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": "allow *" 2024-09-13T17:44:03.375 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-13T17:44:03.375 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-13T17:44:03.375 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-13T17:44:03.375 INFO:teuthology.orchestra.run.smithi102.stdout: "entity": "osd.3", 2024-09-13T17:44:03.375 INFO:teuthology.orchestra.run.smithi102.stdout: "key": "AQBMeeRmahZPOhAA9ni0BvesPpGsHJAof7+Psg==", 2024-09-13T17:44:03.375 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": { 2024-09-13T17:44:03.375 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": "allow profile osd", 2024-09-13T17:44:03.376 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": "allow profile osd", 2024-09-13T17:44:03.376 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": "allow *" 2024-09-13T17:44:03.376 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-13T17:44:03.376 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-13T17:44:03.376 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-13T17:44:03.376 INFO:teuthology.orchestra.run.smithi102.stdout: "entity": "client.0", 2024-09-13T17:44:03.376 INFO:teuthology.orchestra.run.smithi102.stdout: "key": "AQBAeeRmt/H1KxAAHEY0kvAg5ouvDN3LtMj2KA==", 2024-09-13T17:44:03.376 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": { 2024-09-13T17:44:03.376 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": "allow", 2024-09-13T17:44:03.376 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": "allow r", 2024-09-13T17:44:03.376 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": "allow rw", 2024-09-13T17:44:03.376 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": "allow rwx" 2024-09-13T17:44:03.376 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-13T17:44:03.376 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-13T17:44:03.377 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-13T17:44:03.377 INFO:teuthology.orchestra.run.smithi102.stdout: "entity": "client.admin", 2024-09-13T17:44:03.377 INFO:teuthology.orchestra.run.smithi102.stdout: "key": "AQBAeeRmjZoaEBAAX/Pq2+o1m9kMekGxN8MKLA==", 2024-09-13T17:44:03.377 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": { 2024-09-13T17:44:03.377 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": "allow *", 2024-09-13T17:44:03.377 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": "allow *", 2024-09-13T17:44:03.377 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": "allow *", 2024-09-13T17:44:03.377 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": "allow *" 2024-09-13T17:44:03.377 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-13T17:44:03.377 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-13T17:44:03.377 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-13T17:44:03.377 INFO:teuthology.orchestra.run.smithi102.stdout: "entity": "client.bootstrap-mds", 2024-09-13T17:44:03.377 INFO:teuthology.orchestra.run.smithi102.stdout: "key": "AQBWeeRmgvT2IxAAEFEuq44UWpLN+seCMz75rg==", 2024-09-13T17:44:03.377 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": { 2024-09-13T17:44:03.378 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": "allow profile bootstrap-mds" 2024-09-13T17:44:03.378 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-13T17:44:03.378 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-13T17:44:03.378 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-13T17:44:03.378 INFO:teuthology.orchestra.run.smithi102.stdout: "entity": "client.bootstrap-mgr", 2024-09-13T17:44:03.378 INFO:teuthology.orchestra.run.smithi102.stdout: "key": "AQBWeeRmkf72IxAALOI9sS+FSNHNF2u7FkwPHg==", 2024-09-13T17:44:03.378 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": { 2024-09-13T17:44:03.378 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-13T17:44:03.378 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-13T17:44:03.378 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-13T17:44:03.378 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-13T17:44:03.378 INFO:teuthology.orchestra.run.smithi102.stdout: "entity": "client.bootstrap-osd", 2024-09-13T17:44:03.378 INFO:teuthology.orchestra.run.smithi102.stdout: "key": "AQBWeeRm1gX3IxAAuhVscS7yz4GviKd9RS2P8Q==", 2024-09-13T17:44:03.378 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": { 2024-09-13T17:44:03.379 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": "allow profile bootstrap-osd" 2024-09-13T17:44:03.379 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-13T17:44:03.379 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-13T17:44:03.379 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-13T17:44:03.379 INFO:teuthology.orchestra.run.smithi102.stdout: "entity": "client.bootstrap-rbd", 2024-09-13T17:44:03.379 INFO:teuthology.orchestra.run.smithi102.stdout: "key": "AQBWeeRmrQz3IxAAdflGadyrb6iSMmdGaqp/8Q==", 2024-09-13T17:44:03.379 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": { 2024-09-13T17:44:03.379 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-13T17:44:03.379 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-13T17:44:03.379 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-13T17:44:03.379 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-13T17:44:03.379 INFO:teuthology.orchestra.run.smithi102.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-13T17:44:03.379 INFO:teuthology.orchestra.run.smithi102.stdout: "key": "AQBWeeRmuhP3IxAAP/DDWA2mW7PXGsNITV3u6A==", 2024-09-13T17:44:03.379 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": { 2024-09-13T17:44:03.380 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-13T17:44:03.380 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-13T17:44:03.380 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-13T17:44:03.380 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-13T17:44:03.380 INFO:teuthology.orchestra.run.smithi102.stdout: "entity": "client.bootstrap-rgw", 2024-09-13T17:44:03.380 INFO:teuthology.orchestra.run.smithi102.stdout: "key": "AQBWeeRmVRz3IxAAXCzV4OzNF/HkUpCUnbFsuA==", 2024-09-13T17:44:03.380 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": { 2024-09-13T17:44:03.380 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-13T17:44:03.380 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-13T17:44:03.380 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-13T17:44:03.380 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-13T17:44:03.380 INFO:teuthology.orchestra.run.smithi102.stdout: "entity": "mgr.x", 2024-09-13T17:44:03.380 INFO:teuthology.orchestra.run.smithi102.stdout: "key": "AQBAeeRm9Zm5JBAA1j3HzJzDG8zz5XNmJ9kTAg==", 2024-09-13T17:44:03.380 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": { 2024-09-13T17:44:03.381 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": "allow *", 2024-09-13T17:44:03.381 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": "allow profile mgr", 2024-09-13T17:44:03.381 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": "allow *" 2024-09-13T17:44:03.381 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-13T17:44:03.381 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-13T17:44:03.381 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-13T17:44:03.381 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-13T17:44:03.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.372+0000 7f2ed299d640 1 -- 172.21.15.102:0/2362934609 >> v1:172.21.15.102:6816/34937 conn(0x7f2eac031160 legacy=0x7f2eac033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:03.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.372+0000 7f2ed299d640 1 -- 172.21.15.102:0/2362934609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ecc09de20 msgr2=0x7f2ecc134620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:03.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.372+0000 7f2ed299d640 1 --2- 172.21.15.102:0/2362934609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ecc09de20 0x7f2ecc134620 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009880 tx=0x7f2eb4032000 comp rx=0 tx=0).stop 2024-09-13T17:44:03.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.372+0000 7f2ed299d640 1 -- 172.21.15.102:0/2362934609 shutdown_connections 2024-09-13T17:44:03.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.372+0000 7f2ed299d640 1 --2- 172.21.15.102:0/2362934609 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ecc09e1f0 0x7f2ecc134b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:03.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.372+0000 7f2ed299d640 1 --2- 172.21.15.102:0/2362934609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ecc09de20 0x7f2ecc134620 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:03.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.372+0000 7f2ed299d640 1 --2- 172.21.15.102:0/2362934609 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ecc068920 0x7f2ecc1340e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:03.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.372+0000 7f2ed299d640 1 -- 172.21.15.102:0/2362934609 >> 172.21.15.102:0/2362934609 conn(0x7f2ecc06f320 msgr2=0x7f2ecc09be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:03.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.372+0000 7f2ed299d640 1 -- 172.21.15.102:0/2362934609 shutdown_connections 2024-09-13T17:44:03.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.372+0000 7f2ed299d640 1 -- 172.21.15.102:0/2362934609 wait complete. 2024-09-13T17:44:03.384 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-13T17:44:03.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.476+0000 7fcefb48b640 1 Processor -- start 2024-09-13T17:44:03.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.477+0000 7fcefb48b640 1 -- start start 2024-09-13T17:44:03.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.477+0000 7fcefb48b640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fcef409abb0 0x7fcef412c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:03.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.477+0000 7fcefb48b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcef412c6a0 0x7fcef412cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:03.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.477+0000 7fcefb48b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcef4096d80 0x7fcef4097200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:03.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.477+0000 7fcefb48b640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef409af80 con 0x7fcef409abb0 2024-09-13T17:44:03.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.477+0000 7fcefb48b640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef412cff0 con 0x7fcef412c6a0 2024-09-13T17:44:03.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.477+0000 7fcefb48b640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcef406a530 con 0x7fcef4096d80 2024-09-13T17:44:03.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.477+0000 7fcef9227640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fcef409abb0 0x7fcef412c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:03.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.477+0000 7fcef8a26640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcef412c6a0 0x7fcef412cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:03.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.477+0000 7fcef9a28640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcef4096d80 0x7fcef4097200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:03.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.477+0000 7fcef8a26640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcef412c6a0 0x7fcef412cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:46886/0 (socket says 172.21.15.102:46886) 2024-09-13T17:44:03.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.477+0000 7fcef9a28640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcef4096d80 0x7fcef4097200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54186/0 (socket says 172.21.15.102:54186) 2024-09-13T17:44:03.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.477+0000 7fcef8a26640 1 -- 172.21.15.102:0/1306830205 learned_addr learned my addr 172.21.15.102:0/1306830205 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:03.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.478+0000 7fcef8a26640 1 -- 172.21.15.102:0/1306830205 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcef4096d80 msgr2=0x7fcef4097200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:03.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.478+0000 7fcef8a26640 1 --2- 172.21.15.102:0/1306830205 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcef4096d80 0x7fcef4097200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:03.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.478+0000 7fcef8a26640 1 -- 172.21.15.102:0/1306830205 >> v2:172.21.15.102:3300/0 conn(0x7fcef409abb0 msgr2=0x7fcef412c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:03.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.478+0000 7fcef8a26640 1 --2- 172.21.15.102:0/1306830205 >> v2:172.21.15.102:3300/0 conn(0x7fcef409abb0 0x7fcef412c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:03.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.478+0000 7fcef8a26640 1 -- 172.21.15.102:0/1306830205 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef4098be0 con 0x7fcef412c6a0 2024-09-13T17:44:03.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.478+0000 7fcef9227640 1 --2- 172.21.15.102:0/1306830205 >> v2:172.21.15.102:3300/0 conn(0x7fcef409abb0 0x7fcef412c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:03.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.478+0000 7fcef9a28640 1 --2- 172.21.15.102:0/1306830205 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcef4096d80 0x7fcef4097200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:03.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.478+0000 7fcef8a26640 1 --2- 172.21.15.102:0/1306830205 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcef412c6a0 0x7fcef412cab0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcee8009910 tx=0x7fcee80091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d2f6911127a6e47 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:03.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.478+0000 7fcef3fff640 1 -- 172.21.15.102:0/1306830205 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee802ac80 con 0x7fcef412c6a0 2024-09-13T17:44:03.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.478+0000 7fcef3fff640 1 -- 172.21.15.102:0/1306830205 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee802ade0 con 0x7fcef412c6a0 2024-09-13T17:44:03.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.478+0000 7fcef3fff640 1 -- 172.21.15.102:0/1306830205 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee8032670 con 0x7fcef412c6a0 2024-09-13T17:44:03.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.479+0000 7fcefb48b640 1 -- 172.21.15.102:0/1306830205 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcef412c6a0 msgr2=0x7fcef412cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:03.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.479+0000 7fcefb48b640 1 --2- 172.21.15.102:0/1306830205 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcef412c6a0 0x7fcef412cab0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcee8009910 tx=0x7fcee80091d0 comp rx=0 tx=0).stop 2024-09-13T17:44:03.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.479+0000 7fcefb48b640 1 -- 172.21.15.102:0/1306830205 shutdown_connections 2024-09-13T17:44:03.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.479+0000 7fcefb48b640 1 --2- 172.21.15.102:0/1306830205 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcef4096d80 0x7fcef4097200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:03.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.479+0000 7fcefb48b640 1 --2- 172.21.15.102:0/1306830205 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcef412c6a0 0x7fcef412cab0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:03.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.479+0000 7fcefb48b640 1 --2- 172.21.15.102:0/1306830205 >> v2:172.21.15.102:3300/0 conn(0x7fcef409abb0 0x7fcef412c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:03.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.479+0000 7fcefb48b640 1 -- 172.21.15.102:0/1306830205 >> 172.21.15.102:0/1306830205 conn(0x7fcef409e820 msgr2=0x7fcef409ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:03.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.479+0000 7fcefb48b640 1 -- 172.21.15.102:0/1306830205 shutdown_connections 2024-09-13T17:44:03.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.479+0000 7fcefb48b640 1 -- 172.21.15.102:0/1306830205 wait complete. 2024-09-13T17:44:03.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.480+0000 7fcefb48b640 1 Processor -- start 2024-09-13T17:44:03.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.480+0000 7fcefb48b640 1 -- start start 2024-09-13T17:44:03.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.480+0000 7fcefb48b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcef4096d80 0x7fcef41178a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:03.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.480+0000 7fcefb48b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcef409abb0 0x7fcef4117de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:03.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.480+0000 7fcefb48b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcef412c6a0 0x7fcef413c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:03.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.480+0000 7fcefb48b640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef40981a0 con 0x7fcef412c6a0 2024-09-13T17:44:03.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.480+0000 7fcefb48b640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef41199c0 con 0x7fcef4096d80 2024-09-13T17:44:03.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.480+0000 7fcefb48b640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcef411a8e0 con 0x7fcef409abb0 2024-09-13T17:44:03.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.481+0000 7fcef9227640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcef4096d80 0x7fcef41178a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:03.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.481+0000 7fcef8a26640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcef409abb0 0x7fcef4117de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:03.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.481+0000 7fcef9227640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcef4096d80 0x7fcef41178a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:46888/0 (socket says 172.21.15.102:46888) 2024-09-13T17:44:03.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.481+0000 7fcef9227640 1 -- 172.21.15.102:0/1185113628 learned_addr learned my addr 172.21.15.102:0/1185113628 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:03.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.481+0000 7fcef9a28640 1 --2- 172.21.15.102:0/1185113628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcef412c6a0 0x7fcef413c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:03.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.481+0000 7fcef8a26640 1 -- 172.21.15.102:0/1185113628 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcef4096d80 msgr2=0x7fcef41178a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:03.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.481+0000 7fcef8a26640 1 --2- 172.21.15.102:0/1185113628 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcef4096d80 0x7fcef41178a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:03.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.481+0000 7fcef8a26640 1 -- 172.21.15.102:0/1185113628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcef412c6a0 msgr2=0x7fcef413c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:03.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.481+0000 7fcef8a26640 1 --2- 172.21.15.102:0/1185113628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcef412c6a0 0x7fcef413c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:03.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.481+0000 7fcef8a26640 1 -- 172.21.15.102:0/1185113628 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee4009a20 con 0x7fcef409abb0 2024-09-13T17:44:03.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.481+0000 7fcef9a28640 1 --2- 172.21.15.102:0/1185113628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcef412c6a0 0x7fcef413c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:03.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.481+0000 7fcef9227640 1 --2- 172.21.15.102:0/1185113628 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcef4096d80 0x7fcef41178a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:03.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.481+0000 7fcef8a26640 1 --2- 172.21.15.102:0/1185113628 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcef409abb0 0x7fcef4117de0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcee8031000 tx=0x7fcee8032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:03.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.481+0000 7fcef17fa640 1 -- 172.21.15.102:0/1185113628 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee803d630 con 0x7fcef409abb0 2024-09-13T17:44:03.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.481+0000 7fcef17fa640 1 -- 172.21.15.102:0/1185113628 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee803d7b0 con 0x7fcef409abb0 2024-09-13T17:44:03.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.481+0000 7fcefb48b640 1 -- 172.21.15.102:0/1185113628 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceec009390 con 0x7fcef409abb0 2024-09-13T17:44:03.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.481+0000 7fcef17fa640 1 -- 172.21.15.102:0/1185113628 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee803dbb0 con 0x7fcef409abb0 2024-09-13T17:44:03.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.481+0000 7fcefb48b640 1 -- 172.21.15.102:0/1185113628 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee8009450 con 0x7fcef409abb0 2024-09-13T17:44:03.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.482+0000 7fcef17fa640 1 -- 172.21.15.102:0/1185113628 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fcee8039040 con 0x7fcef409abb0 2024-09-13T17:44:03.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.482+0000 7fcef17fa640 1 -- 172.21.15.102:0/1185113628 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 2808+0+0 (secure 0 0 0) 0x7fcee8061580 con 0x7fcef409abb0 2024-09-13T17:44:03.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.482+0000 7fcefb48b640 1 -- 172.21.15.102:0/1185113628 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef4117660 con 0x7fcef409abb0 2024-09-13T17:44:03.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.486+0000 7fcef17fa640 1 -- 172.21.15.102:0/1185113628 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fcef4117660 con 0x7fcef409abb0 2024-09-13T17:44:03.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.698+0000 7fcefb48b640 1 -- 172.21.15.102:0/1185113628 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fcef4117660 con 0x7fcef409abb0 2024-09-13T17:44:03.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.701+0000 7fcef17fa640 1 -- 172.21.15.102:0/1185113628 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) v1 ==== 90+0+0 (secure 0 0 0) 0x7fcef4117660 con 0x7fcef409abb0 2024-09-13T17:44:03.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.703+0000 7fcefb48b640 1 -- 172.21.15.102:0/1185113628 >> v1:172.21.15.102:6816/34937 conn(0x7fcee0031160 legacy=0x7fcee0033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:03.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.703+0000 7fcefb48b640 1 -- 172.21.15.102:0/1185113628 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcef409abb0 msgr2=0x7fcef4117de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:03.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.703+0000 7fcefb48b640 1 --2- 172.21.15.102:0/1185113628 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcef409abb0 0x7fcef4117de0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcee8031000 tx=0x7fcee8032000 comp rx=0 tx=0).stop 2024-09-13T17:44:03.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.703+0000 7fcefb48b640 1 -- 172.21.15.102:0/1185113628 shutdown_connections 2024-09-13T17:44:03.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.703+0000 7fcefb48b640 1 --2- 172.21.15.102:0/1185113628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcef412c6a0 0x7fcef413c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:03.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.703+0000 7fcefb48b640 1 --2- 172.21.15.102:0/1185113628 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcef409abb0 0x7fcef4117de0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:03.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.703+0000 7fcefb48b640 1 --2- 172.21.15.102:0/1185113628 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcef4096d80 0x7fcef41178a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:03.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.703+0000 7fcefb48b640 1 -- 172.21.15.102:0/1185113628 >> 172.21.15.102:0/1185113628 conn(0x7fcef409e820 msgr2=0x7fcef412f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:03.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.703+0000 7fcefb48b640 1 -- 172.21.15.102:0/1185113628 shutdown_connections 2024-09-13T17:44:03.705 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.703+0000 7fcefb48b640 1 -- 172.21.15.102:0/1185113628 wait complete. 2024-09-13T17:44:03.715 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-13T17:44:03.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.807+0000 7f4beeb9d640 1 Processor -- start 2024-09-13T17:44:03.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.807+0000 7f4beeb9d640 1 -- start start 2024-09-13T17:44:03.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.808+0000 7f4beeb9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4be806b140 0x7f4be80998c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:03.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.808+0000 7f4beeb9d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4be81325b0 0x7f4be8134990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:03.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.808+0000 7f4beeb9d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be806ee90 con 0x7f4be809ac20 2024-09-13T17:44:03.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.808+0000 7f4beeb9d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be806ac90 con 0x7f4be81325b0 2024-09-13T17:44:03.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.808+0000 7f4beeb9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4be809a510 con 0x7f4be806b140 2024-09-13T17:44:03.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.808+0000 7f4bec939640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4be806b140 0x7f4be80998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:03.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.808+0000 7f4bed13a640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4be81325b0 0x7f4be8134990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:03.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.808+0000 7f4bed13a640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f4be81325b0 0x7f4be8134990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34254/0 (socket says 172.21.15.102:34254) 2024-09-13T17:44:03.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.808+0000 7f4bec939640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4be806b140 0x7f4be80998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54206/0 (socket says 172.21.15.102:54206) 2024-09-13T17:44:03.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.808+0000 7f4bed13a640 1 -- 172.21.15.102:0/2987883431 learned_addr learned my addr 172.21.15.102:0/2987883431 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:03.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.808+0000 7f4bec939640 1 -- 172.21.15.102:0/2987883431 >> v2:172.21.15.102:3300/0 conn(0x7f4be81325b0 msgr2=0x7f4be8134990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:03.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.808+0000 7f4bec939640 1 --2- 172.21.15.102:0/2987883431 >> v2:172.21.15.102:3300/0 conn(0x7f4be81325b0 0x7f4be8134990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:03.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.808+0000 7f4bec939640 1 -- 172.21.15.102:0/2987883431 >> v1:172.21.15.102:6789/0 conn(0x7f4be809ac20 legacy=0x7f4be8099e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:03.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.808+0000 7f4bec939640 1 -- 172.21.15.102:0/2987883431 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be8099510 con 0x7f4be806b140 2024-09-13T17:44:03.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.808+0000 7f4bed13a640 1 --2- 172.21.15.102:0/2987883431 >> v2:172.21.15.102:3300/0 conn(0x7f4be81325b0 0x7f4be8134990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:03.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.809+0000 7f4bec939640 1 --2- 172.21.15.102:0/2987883431 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4be806b140 0x7f4be80998c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8009850 tx=0x7f4bd80090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ed01dcbc76a5f01 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:03.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.809+0000 7f4be77fe640 1 -- 172.21.15.102:0/2987883431 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd802ac80 con 0x7f4be806b140 2024-09-13T17:44:03.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.809+0000 7f4be77fe640 1 -- 172.21.15.102:0/2987883431 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd802ade0 con 0x7f4be806b140 2024-09-13T17:44:03.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.809+0000 7f4be77fe640 1 -- 172.21.15.102:0/2987883431 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8032670 con 0x7f4be806b140 2024-09-13T17:44:03.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.809+0000 7f4beeb9d640 1 -- 172.21.15.102:0/2987883431 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4be806b140 msgr2=0x7f4be80998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:03.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.809+0000 7f4beeb9d640 1 --2- 172.21.15.102:0/2987883431 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4be806b140 0x7f4be80998c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8009850 tx=0x7f4bd80090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:03.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.810+0000 7f4beeb9d640 1 -- 172.21.15.102:0/2987883431 shutdown_connections 2024-09-13T17:44:03.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.810+0000 7f4beeb9d640 1 --2- 172.21.15.102:0/2987883431 >> v2:172.21.15.102:3300/0 conn(0x7f4be81325b0 0x7f4be8134990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:03.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.810+0000 7f4beeb9d640 1 --2- 172.21.15.102:0/2987883431 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4be806b140 0x7f4be80998c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:03.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.810+0000 7f4beeb9d640 1 -- 172.21.15.102:0/2987883431 >> 172.21.15.102:0/2987883431 conn(0x7f4be809e820 msgr2=0x7f4be809ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:03.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.810+0000 7f4beeb9d640 1 -- 172.21.15.102:0/2987883431 shutdown_connections 2024-09-13T17:44:03.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.810+0000 7f4beeb9d640 1 -- 172.21.15.102:0/2987883431 wait complete. 2024-09-13T17:44:03.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.810+0000 7f4beeb9d640 1 Processor -- start 2024-09-13T17:44:03.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.811+0000 7f4beeb9d640 1 -- start start 2024-09-13T17:44:03.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.811+0000 7f4beeb9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4be806b140 0x7f4be8120110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:03.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.811+0000 7f4beeb9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4be809ac20 0x7f4be8096a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:03.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.811+0000 7f4beeb9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4be81325b0 0x7f4be8096fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:03.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.811+0000 7f4beeb9d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be8097ee0 con 0x7f4be81325b0 2024-09-13T17:44:03.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.811+0000 7f4beeb9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be811f930 con 0x7f4be809ac20 2024-09-13T17:44:03.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.811+0000 7f4beeb9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4be81348b0 con 0x7f4be806b140 2024-09-13T17:44:03.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.811+0000 7f4bec939640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4be806b140 0x7f4be8120110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:03.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.811+0000 7f4bed13a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4be81325b0 0x7f4be8096fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:03.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.811+0000 7f4bec939640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4be806b140 0x7f4be8120110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54212/0 (socket says 172.21.15.102:54212) 2024-09-13T17:44:03.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.811+0000 7f4bec939640 1 -- 172.21.15.102:0/2733143404 learned_addr learned my addr 172.21.15.102:0/2733143404 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:03.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.811+0000 7f4be7fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4be809ac20 0x7f4be8096a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:03.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.811+0000 7f4bec939640 1 -- 172.21.15.102:0/2733143404 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4be809ac20 msgr2=0x7f4be8096a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:03.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.811+0000 7f4bec939640 1 --2- 172.21.15.102:0/2733143404 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4be809ac20 0x7f4be8096a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:03.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.811+0000 7f4bec939640 1 -- 172.21.15.102:0/2733143404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4be81325b0 msgr2=0x7f4be8096fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:03.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.811+0000 7f4bec939640 1 --2- 172.21.15.102:0/2733143404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4be81325b0 0x7f4be8096fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:03.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.811+0000 7f4bec939640 1 -- 172.21.15.102:0/2733143404 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be0009ae0 con 0x7f4be806b140 2024-09-13T17:44:03.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.812+0000 7f4bec939640 1 --2- 172.21.15.102:0/2733143404 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4be806b140 0x7f4be8120110 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8009850 tx=0x7f4bd8032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:03.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.812+0000 7f4be4ff9640 1 -- 172.21.15.102:0/2733143404 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd803d720 con 0x7f4be806b140 2024-09-13T17:44:03.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.812+0000 7f4be4ff9640 1 -- 172.21.15.102:0/2733143404 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd803d880 con 0x7f4be806b140 2024-09-13T17:44:03.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.812+0000 7f4beeb9d640 1 -- 172.21.15.102:0/2733143404 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd8009390 con 0x7f4be806b140 2024-09-13T17:44:03.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.812+0000 7f4be4ff9640 1 -- 172.21.15.102:0/2733143404 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd803dc00 con 0x7f4be806b140 2024-09-13T17:44:03.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.812+0000 7f4beeb9d640 1 -- 172.21.15.102:0/2733143404 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be80985d0 con 0x7f4be806b140 2024-09-13T17:44:03.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.812+0000 7f4be4ff9640 1 -- 172.21.15.102:0/2733143404 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f4bd8039040 con 0x7f4be806b140 2024-09-13T17:44:03.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.813+0000 7f4be4ff9640 1 -- 172.21.15.102:0/2733143404 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 2808+0+0 (secure 0 0 0) 0x7f4bd8061790 con 0x7f4be806b140 2024-09-13T17:44:03.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.813+0000 7f4beeb9d640 1 -- 172.21.15.102:0/2733143404 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be8064000 con 0x7f4be806b140 2024-09-13T17:44:03.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:03.817+0000 7f4be4ff9640 1 -- 172.21.15.102:0/2733143404 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f4bd8037080 con 0x7f4be806b140 2024-09-13T17:44:04.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.026+0000 7f4beeb9d640 1 -- 172.21.15.102:0/2733143404 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f4be8116550 con 0x7f4be806b140 2024-09-13T17:44:04.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.029+0000 7f4be4ff9640 1 -- 172.21.15.102:0/2733143404 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7f4bd8042020 con 0x7f4be806b140 2024-09-13T17:44:04.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.031+0000 7f4beeb9d640 1 -- 172.21.15.102:0/2733143404 >> v1:172.21.15.102:6816/34937 conn(0x7f4bd4031160 legacy=0x7f4bd4033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:04.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.031+0000 7f4beeb9d640 1 -- 172.21.15.102:0/2733143404 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4be806b140 msgr2=0x7f4be8120110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:04.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.031+0000 7f4beeb9d640 1 --2- 172.21.15.102:0/2733143404 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4be806b140 0x7f4be8120110 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8009850 tx=0x7f4bd8032000 comp rx=0 tx=0).stop 2024-09-13T17:44:04.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.031+0000 7f4beeb9d640 1 -- 172.21.15.102:0/2733143404 shutdown_connections 2024-09-13T17:44:04.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.031+0000 7f4beeb9d640 1 --2- 172.21.15.102:0/2733143404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4be81325b0 0x7f4be8096fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:04.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.031+0000 7f4beeb9d640 1 --2- 172.21.15.102:0/2733143404 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4be809ac20 0x7f4be8096a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:04.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.031+0000 7f4beeb9d640 1 --2- 172.21.15.102:0/2733143404 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4be806b140 0x7f4be8120110 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:04.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.031+0000 7f4beeb9d640 1 -- 172.21.15.102:0/2733143404 >> 172.21.15.102:0/2733143404 conn(0x7f4be809e820 msgr2=0x7f4be806e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:04.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.031+0000 7f4beeb9d640 1 -- 172.21.15.102:0/2733143404 shutdown_connections 2024-09-13T17:44:04.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.031+0000 7f4beeb9d640 1 -- 172.21.15.102:0/2733143404 wait complete. 2024-09-13T17:44:04.043 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-13T17:44:04.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.136+0000 7f63f3f5d640 1 Processor -- start 2024-09-13T17:44:04.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.137+0000 7f63f3f5d640 1 -- start start 2024-09-13T17:44:04.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.137+0000 7f63f3f5d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63ec09abb0 0x7f63ec12c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:04.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.137+0000 7f63f3f5d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f63ec12cab0 0x7f63ec097490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:04.140 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.137+0000 7f63f3f5d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63ec06a530 con 0x7f63ec12c6a0 2024-09-13T17:44:04.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.137+0000 7f63f3f5d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63ec09af80 con 0x7f63ec12cab0 2024-09-13T17:44:04.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.137+0000 7f63f3f5d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec098be0 con 0x7f63ec09abb0 2024-09-13T17:44:04.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.137+0000 7f63f1cf9640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63ec09abb0 0x7f63ec12c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:04.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.137+0000 7f63f24fa640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f63ec12cab0 0x7f63ec097490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:04.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.137+0000 7f63f24fa640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f63ec12cab0 0x7f63ec097490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34280/0 (socket says 172.21.15.102:34280) 2024-09-13T17:44:04.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.137+0000 7f63f14f8640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f63ec12c6a0 0x7f63ec096d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:37404/0 (socket says 172.21.15.102:37404) 2024-09-13T17:44:04.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.137+0000 7f63f14f8640 1 -- 172.21.15.102:0/3181203092 learned_addr learned my addr 172.21.15.102:0/3181203092 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:44:04.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.137+0000 7f63f1cf9640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63ec09abb0 0x7f63ec12c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:46908/0 (socket says 172.21.15.102:46908) 2024-09-13T17:44:04.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.138+0000 7f63f24fa640 1 -- 172.21.15.102:0/3181203092 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63ec09abb0 msgr2=0x7f63ec12c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:04.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.138+0000 7f63f24fa640 1 --2- 172.21.15.102:0/3181203092 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63ec09abb0 0x7f63ec12c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:04.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.138+0000 7f63f24fa640 1 -- 172.21.15.102:0/3181203092 >> v1:172.21.15.102:6789/0 conn(0x7f63ec12c6a0 legacy=0x7f63ec096d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:04.141 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.138+0000 7f63f0cf7640 1 -- 172.21.15.102:0/3181203092 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3646145992 0 0) 0x7f63ec06a530 con 0x7f63ec12c6a0 2024-09-13T17:44:04.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.138+0000 7f63f24fa640 1 -- 172.21.15.102:0/3181203092 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ec098930 con 0x7f63ec12cab0 2024-09-13T17:44:04.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.138+0000 7f63f1cf9640 1 --2- 172.21.15.102:0/3181203092 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63ec09abb0 0x7f63ec12c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:04.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.138+0000 7f63f24fa640 1 --2- 172.21.15.102:0/3181203092 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63ec12cab0 0x7f63ec097490 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f63e4009e60 tx=0x7f63e4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae905fec56d4801d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:04.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.138+0000 7f63f0cf7640 1 -- 172.21.15.102:0/3181203092 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e4007550 con 0x7f63ec12cab0 2024-09-13T17:44:04.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.138+0000 7f63f0cf7640 1 -- 172.21.15.102:0/3181203092 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e402d050 con 0x7f63ec12cab0 2024-09-13T17:44:04.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.138+0000 7f63f0cf7640 1 -- 172.21.15.102:0/3181203092 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e40078e0 con 0x7f63ec12cab0 2024-09-13T17:44:04.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.139+0000 7f63f3f5d640 1 -- 172.21.15.102:0/3181203092 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63ec12cab0 msgr2=0x7f63ec097490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:04.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.139+0000 7f63f3f5d640 1 --2- 172.21.15.102:0/3181203092 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63ec12cab0 0x7f63ec097490 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f63e4009e60 tx=0x7f63e4009780 comp rx=0 tx=0).stop 2024-09-13T17:44:04.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.139+0000 7f63f3f5d640 1 -- 172.21.15.102:0/3181203092 shutdown_connections 2024-09-13T17:44:04.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.139+0000 7f63f3f5d640 1 --2- 172.21.15.102:0/3181203092 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63ec12cab0 0x7f63ec097490 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:04.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.139+0000 7f63f3f5d640 1 --2- 172.21.15.102:0/3181203092 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63ec09abb0 0x7f63ec12c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:04.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.139+0000 7f63f3f5d640 1 -- 172.21.15.102:0/3181203092 >> 172.21.15.102:0/3181203092 conn(0x7f63ec09e820 msgr2=0x7f63ec09ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:04.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.139+0000 7f63f3f5d640 1 -- 172.21.15.102:0/3181203092 shutdown_connections 2024-09-13T17:44:04.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.139+0000 7f63f3f5d640 1 -- 172.21.15.102:0/3181203092 wait complete. 2024-09-13T17:44:04.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.140+0000 7f63f3f5d640 1 Processor -- start 2024-09-13T17:44:04.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.140+0000 7f63f3f5d640 1 -- start start 2024-09-13T17:44:04.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.140+0000 7f63f3f5d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f63ec09abb0 0x7f63ec119ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:04.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.140+0000 7f63f3f5d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63ec12c6a0 0x7f63ec11a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:04.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.140+0000 7f63f3f5d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63ec12cab0 0x7f63ec11a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:04.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.140+0000 7f63f3f5d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec11b6f0 con 0x7f63ec12cab0 2024-09-13T17:44:04.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.140+0000 7f63f3f5d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec11c130 con 0x7f63ec12c6a0 2024-09-13T17:44:04.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.140+0000 7f63f3f5d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec09c8a0 con 0x7f63ec09abb0 2024-09-13T17:44:04.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.140+0000 7f63f14f8640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63ec12c6a0 0x7f63ec11a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:04.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.140+0000 7f63f24fa640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63ec12cab0 0x7f63ec11a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:04.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.140+0000 7f63f1cf9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f63ec09abb0 0x7f63ec119ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:04.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.140+0000 7f63f1cf9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f63ec09abb0 0x7f63ec119ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54224/0 (socket says 172.21.15.102:54224) 2024-09-13T17:44:04.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.140+0000 7f63f1cf9640 1 -- 172.21.15.102:0/4266700809 learned_addr learned my addr 172.21.15.102:0/4266700809 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:04.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.141+0000 7f63f1cf9640 1 -- 172.21.15.102:0/4266700809 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63ec12c6a0 msgr2=0x7f63ec11a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:04.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.141+0000 7f63f1cf9640 1 --2- 172.21.15.102:0/4266700809 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63ec12c6a0 0x7f63ec11a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:04.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.141+0000 7f63f1cf9640 1 -- 172.21.15.102:0/4266700809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63ec12cab0 msgr2=0x7f63ec11a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:04.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.141+0000 7f63f1cf9640 1 --2- 172.21.15.102:0/4266700809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63ec12cab0 0x7f63ec11a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:04.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.141+0000 7f63f1cf9640 1 -- 172.21.15.102:0/4266700809 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e0009390 con 0x7f63ec09abb0 2024-09-13T17:44:04.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.141+0000 7f63f1cf9640 1 --2- 172.21.15.102:0/4266700809 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f63ec09abb0 0x7f63ec119ca0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f63e000ece0 tx=0x7f63e00090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:04.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.141+0000 7f63ddffb640 1 -- 172.21.15.102:0/4266700809 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e000cbc0 con 0x7f63ec09abb0 2024-09-13T17:44:04.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.141+0000 7f63ddffb640 1 -- 172.21.15.102:0/4266700809 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e000cd20 con 0x7f63ec09abb0 2024-09-13T17:44:04.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.141+0000 7f63ddffb640 1 -- 172.21.15.102:0/4266700809 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e00136a0 con 0x7f63ec09abb0 2024-09-13T17:44:04.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.141+0000 7f63f3f5d640 1 -- 172.21.15.102:0/4266700809 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e4009a20 con 0x7f63ec09abb0 2024-09-13T17:44:04.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.141+0000 7f63f3f5d640 1 -- 172.21.15.102:0/4266700809 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63ec11b1b0 con 0x7f63ec09abb0 2024-09-13T17:44:04.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.142+0000 7f63ddffb640 1 -- 172.21.15.102:0/4266700809 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f63e0013a10 con 0x7f63ec09abb0 2024-09-13T17:44:04.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.142+0000 7f63ddffb640 1 -- 172.21.15.102:0/4266700809 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 2808+0+0 (secure 0 0 0) 0x7f63e0042070 con 0x7f63ec09abb0 2024-09-13T17:44:04.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.142+0000 7f63f3f5d640 1 -- 172.21.15.102:0/4266700809 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63ec11ca60 con 0x7f63ec09abb0 2024-09-13T17:44:04.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.146+0000 7f63ddffb640 1 -- 172.21.15.102:0/4266700809 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f63ec11ca60 con 0x7f63ec09abb0 2024-09-13T17:44:04.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.355+0000 7f63f3f5d640 1 -- 172.21.15.102:0/4266700809 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f63ec11ca60 con 0x7f63ec09abb0 2024-09-13T17:44:04.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.358+0000 7f63ddffb640 1 -- 172.21.15.102:0/4266700809 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f63ec11ca60 con 0x7f63ec09abb0 2024-09-13T17:44:04.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.360+0000 7f63f3f5d640 1 -- 172.21.15.102:0/4266700809 >> v1:172.21.15.102:6816/34937 conn(0x7f63d0031160 legacy=0x7f63d0033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:04.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.360+0000 7f63f3f5d640 1 -- 172.21.15.102:0/4266700809 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f63ec09abb0 msgr2=0x7f63ec119ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:04.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.360+0000 7f63f3f5d640 1 --2- 172.21.15.102:0/4266700809 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f63ec09abb0 0x7f63ec119ca0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f63e000ece0 tx=0x7f63e00090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:04.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.360+0000 7f63f3f5d640 1 -- 172.21.15.102:0/4266700809 shutdown_connections 2024-09-13T17:44:04.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.360+0000 7f63f3f5d640 1 --2- 172.21.15.102:0/4266700809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63ec12cab0 0x7f63ec11a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:04.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.360+0000 7f63f3f5d640 1 --2- 172.21.15.102:0/4266700809 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f63ec12c6a0 0x7f63ec11a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:04.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.360+0000 7f63f3f5d640 1 --2- 172.21.15.102:0/4266700809 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f63ec09abb0 0x7f63ec119ca0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:04.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.360+0000 7f63f3f5d640 1 -- 172.21.15.102:0/4266700809 >> 172.21.15.102:0/4266700809 conn(0x7f63ec09e820 msgr2=0x7f63ec099aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:04.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.360+0000 7f63f3f5d640 1 -- 172.21.15.102:0/4266700809 shutdown_connections 2024-09-13T17:44:04.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.360+0000 7f63f3f5d640 1 -- 172.21.15.102:0/4266700809 wait complete. 2024-09-13T17:44:04.373 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-13T17:44:04.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.466+0000 7f9f59c60640 1 Processor -- start 2024-09-13T17:44:04.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.466+0000 7f9f59c60640 1 -- start start 2024-09-13T17:44:04.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.466+0000 7f9f59c60640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9f54128330 0x7f9f54128700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:04.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.466+0000 7f9f59c60640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9f54128c40 0x7f9f5412b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:04.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.466+0000 7f9f59c60640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9f5412bd80 0x7f9f5412c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:04.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.466+0000 7f9f59c60640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f5412c710 con 0x7f9f54128330 2024-09-13T17:44:04.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.466+0000 7f9f59c60640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f54097be0 con 0x7f9f54128c40 2024-09-13T17:44:04.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.466+0000 7f9f59c60640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f54098e70 con 0x7f9f5412bd80 2024-09-13T17:44:04.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.467+0000 7f9f53fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9f5412bd80 0x7f9f5412c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:04.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.467+0000 7f9f52ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9f54128c40 0x7f9f5412b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:04.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.467+0000 7f9f537fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f9f54128330 0x7f9f54128700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:04.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.467+0000 7f9f53fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9f5412bd80 0x7f9f5412c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54234/0 (socket says 172.21.15.102:54234) 2024-09-13T17:44:04.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.467+0000 7f9f52ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9f54128c40 0x7f9f5412b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:46922/0 (socket says 172.21.15.102:46922) 2024-09-13T17:44:04.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.467+0000 7f9f53fff640 1 -- 172.21.15.102:0/2667355989 learned_addr learned my addr 172.21.15.102:0/2667355989 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:04.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.467+0000 7f9f53fff640 1 -- 172.21.15.102:0/2667355989 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9f54128c40 msgr2=0x7f9f5412b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:04.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.467+0000 7f9f53fff640 1 --2- 172.21.15.102:0/2667355989 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9f54128c40 0x7f9f5412b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:04.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.467+0000 7f9f53fff640 1 -- 172.21.15.102:0/2667355989 >> v2:172.21.15.102:3300/0 conn(0x7f9f54128330 msgr2=0x7f9f54128700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:04.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.467+0000 7f9f53fff640 1 --2- 172.21.15.102:0/2667355989 >> v2:172.21.15.102:3300/0 conn(0x7f9f54128330 0x7f9f54128700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:04.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.467+0000 7f9f53fff640 1 -- 172.21.15.102:0/2667355989 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f54098310 con 0x7f9f5412bd80 2024-09-13T17:44:04.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.467+0000 7f9f52ffd640 1 --2- 172.21.15.102:0/2667355989 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9f54128c40 0x7f9f5412b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:04.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.467+0000 7f9f537fe640 1 --2- 172.21.15.102:0/2667355989 >> v2:172.21.15.102:3300/0 conn(0x7f9f54128330 0x7f9f54128700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:04.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.468+0000 7f9f53fff640 1 --2- 172.21.15.102:0/2667355989 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9f5412bd80 0x7f9f5412c1d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00d1a0 tx=0x7f9f4c00ba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ef4dde14054f647 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:04.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.468+0000 7f9f527fc640 1 -- 172.21.15.102:0/2667355989 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c034450 con 0x7f9f5412bd80 2024-09-13T17:44:04.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.468+0000 7f9f527fc640 1 -- 172.21.15.102:0/2667355989 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4c0345b0 con 0x7f9f5412bd80 2024-09-13T17:44:04.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.468+0000 7f9f527fc640 1 -- 172.21.15.102:0/2667355989 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c034990 con 0x7f9f5412bd80 2024-09-13T17:44:04.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.468+0000 7f9f59c60640 1 -- 172.21.15.102:0/2667355989 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9f5412bd80 msgr2=0x7f9f5412c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:04.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.468+0000 7f9f59c60640 1 --2- 172.21.15.102:0/2667355989 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9f5412bd80 0x7f9f5412c1d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00d1a0 tx=0x7f9f4c00ba50 comp rx=0 tx=0).stop 2024-09-13T17:44:04.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.468+0000 7f9f59c60640 1 -- 172.21.15.102:0/2667355989 shutdown_connections 2024-09-13T17:44:04.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.468+0000 7f9f59c60640 1 --2- 172.21.15.102:0/2667355989 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9f5412bd80 0x7f9f5412c1d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:04.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.468+0000 7f9f59c60640 1 --2- 172.21.15.102:0/2667355989 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9f54128c40 0x7f9f5412b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:04.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.468+0000 7f9f59c60640 1 --2- 172.21.15.102:0/2667355989 >> v2:172.21.15.102:3300/0 conn(0x7f9f54128330 0x7f9f54128700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:04.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.468+0000 7f9f59c60640 1 -- 172.21.15.102:0/2667355989 >> 172.21.15.102:0/2667355989 conn(0x7f9f5409e820 msgr2=0x7f9f5409ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:04.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.468+0000 7f9f59c60640 1 -- 172.21.15.102:0/2667355989 shutdown_connections 2024-09-13T17:44:04.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.468+0000 7f9f59c60640 1 -- 172.21.15.102:0/2667355989 wait complete. 2024-09-13T17:44:04.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.469+0000 7f9f59c60640 1 Processor -- start 2024-09-13T17:44:04.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.469+0000 7f9f59c60640 1 -- start start 2024-09-13T17:44:04.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.470+0000 7f9f59c60640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f54128330 0x7f9f54114e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:04.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.470+0000 7f9f59c60640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9f54128c40 0x7f9f541153c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:04.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.470+0000 7f9f59c60640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9f5412bd80 0x7f9f54115900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:04.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.470+0000 7f9f59c60640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5406e160 con 0x7f9f54128330 2024-09-13T17:44:04.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.470+0000 7f9f59c60640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f54117080 con 0x7f9f5412bd80 2024-09-13T17:44:04.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.470+0000 7f9f59c60640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f54125fa0 con 0x7f9f54128c40 2024-09-13T17:44:04.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.470+0000 7f9f537fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f54128330 0x7f9f54114e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:04.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.470+0000 7f9f53fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9f5412bd80 0x7f9f54115900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:04.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.470+0000 7f9f52ffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9f54128c40 0x7f9f541153c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:04.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.470+0000 7f9f537fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f54128330 0x7f9f54114e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34304/0 (socket says 172.21.15.102:34304) 2024-09-13T17:44:04.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.470+0000 7f9f52ffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9f54128c40 0x7f9f541153c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54238/0 (socket says 172.21.15.102:54238) 2024-09-13T17:44:04.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.470+0000 7f9f537fe640 1 -- 172.21.15.102:0/1275379429 learned_addr learned my addr 172.21.15.102:0/1275379429 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:04.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.470+0000 7f9f537fe640 1 -- 172.21.15.102:0/1275379429 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9f54128c40 msgr2=0x7f9f541153c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:04.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.470+0000 7f9f537fe640 1 --2- 172.21.15.102:0/1275379429 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9f54128c40 0x7f9f541153c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:04.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.470+0000 7f9f537fe640 1 -- 172.21.15.102:0/1275379429 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9f5412bd80 msgr2=0x7f9f54115900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:04.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.470+0000 7f9f537fe640 1 --2- 172.21.15.102:0/1275379429 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9f5412bd80 0x7f9f54115900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:04.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.470+0000 7f9f537fe640 1 -- 172.21.15.102:0/1275379429 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f44009390 con 0x7f9f54128330 2024-09-13T17:44:04.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.470+0000 7f9f52ffd640 1 --2- 172.21.15.102:0/1275379429 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9f54128c40 0x7f9f541153c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:04.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.470+0000 7f9f53fff640 1 --2- 172.21.15.102:0/1275379429 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9f5412bd80 0x7f9f54115900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:04.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.470+0000 7f9f537fe640 1 --2- 172.21.15.102:0/1275379429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f54128330 0x7f9f54114e80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400ea70 tx=0x7f9f440090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:04.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.470+0000 7f9f3b7fe640 1 -- 172.21.15.102:0/1275379429 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4400ca10 con 0x7f9f54128330 2024-09-13T17:44:04.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.470+0000 7f9f3b7fe640 1 -- 172.21.15.102:0/1275379429 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4400cb70 con 0x7f9f54128330 2024-09-13T17:44:04.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.471+0000 7f9f59c60640 1 -- 172.21.15.102:0/1275379429 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f48009390 con 0x7f9f54128330 2024-09-13T17:44:04.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.471+0000 7f9f3b7fe640 1 -- 172.21.15.102:0/1275379429 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f44013470 con 0x7f9f54128330 2024-09-13T17:44:04.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.471+0000 7f9f59c60640 1 -- 172.21.15.102:0/1275379429 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f4c00bcf0 con 0x7f9f54128330 2024-09-13T17:44:04.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.471+0000 7f9f3b7fe640 1 -- 172.21.15.102:0/1275379429 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f9f44013820 con 0x7f9f54128330 2024-09-13T17:44:04.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.471+0000 7f9f3b7fe640 1 -- 172.21.15.102:0/1275379429 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 2808+0+0 (secure 0 0 0) 0x7f9f440420c0 con 0x7f9f54128330 2024-09-13T17:44:04.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.472+0000 7f9f59c60640 1 -- 172.21.15.102:0/1275379429 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f54064000 con 0x7f9f54128330 2024-09-13T17:44:04.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.476+0000 7f9f3b7fe640 1 -- 172.21.15.102:0/1275379429 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f9f44023ad0 con 0x7f9f54128330 2024-09-13T17:44:04.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.688+0000 7f9f59c60640 1 -- 172.21.15.102:0/1275379429 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f9f5409e2f0 con 0x7f9f54128330 2024-09-13T17:44:04.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.691+0000 7f9f3b7fe640 1 -- 172.21.15.102:0/1275379429 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9f440232f0 con 0x7f9f54128330 2024-09-13T17:44:04.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.693+0000 7f9f59c60640 1 -- 172.21.15.102:0/1275379429 >> v1:172.21.15.102:6816/34937 conn(0x7f9f40031160 legacy=0x7f9f40033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:04.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.693+0000 7f9f59c60640 1 -- 172.21.15.102:0/1275379429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f54128330 msgr2=0x7f9f54114e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:04.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.693+0000 7f9f59c60640 1 --2- 172.21.15.102:0/1275379429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f54128330 0x7f9f54114e80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400ea70 tx=0x7f9f440090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:04.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.693+0000 7f9f59c60640 1 -- 172.21.15.102:0/1275379429 shutdown_connections 2024-09-13T17:44:04.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.693+0000 7f9f59c60640 1 --2- 172.21.15.102:0/1275379429 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9f5412bd80 0x7f9f54115900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:04.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.693+0000 7f9f59c60640 1 --2- 172.21.15.102:0/1275379429 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9f54128c40 0x7f9f541153c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:04.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.693+0000 7f9f59c60640 1 --2- 172.21.15.102:0/1275379429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f54128330 0x7f9f54114e80 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:04.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.693+0000 7f9f59c60640 1 -- 172.21.15.102:0/1275379429 >> 172.21.15.102:0/1275379429 conn(0x7f9f5409e820 msgr2=0x7f9f5412afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:04.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.693+0000 7f9f59c60640 1 -- 172.21.15.102:0/1275379429 shutdown_connections 2024-09-13T17:44:04.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.693+0000 7f9f59c60640 1 -- 172.21.15.102:0/1275379429 wait complete. 2024-09-13T17:44:04.706 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-13T17:44:04.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.798+0000 7fa5c6d9d640 1 Processor -- start 2024-09-13T17:44:04.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.798+0000 7fa5c6d9d640 1 -- start start 2024-09-13T17:44:04.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.798+0000 7fa5c6d9d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa5c0128330 0x7fa5c0128700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:04.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.798+0000 7fa5c6d9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa5c012bf50 0x7fa5c0133b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:04.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.798+0000 7fa5c6d9d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5c0097be0 con 0x7fa5c0128c40 2024-09-13T17:44:04.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.798+0000 7fa5c6d9d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5c0098e70 con 0x7fa5c0128330 2024-09-13T17:44:04.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.798+0000 7fa5c6d9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c0098310 con 0x7fa5c012bf50 2024-09-13T17:44:04.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.799+0000 7fa5c533a640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa5c012bf50 0x7fa5c0133b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:04.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.799+0000 7fa5c4b39640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa5c0128330 0x7fa5c0128700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:04.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.799+0000 7fa5bffff640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fa5c0128c40 0x7fa5c012b840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:37408/0 (socket says 172.21.15.102:37408) 2024-09-13T17:44:04.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.799+0000 7fa5bffff640 1 -- 172.21.15.102:0/2589051552 learned_addr learned my addr 172.21.15.102:0/2589051552 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:44:04.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.799+0000 7fa5c533a640 1 -- 172.21.15.102:0/2589051552 >> v2:172.21.15.102:3300/0 conn(0x7fa5c0128330 msgr2=0x7fa5c0128700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:04.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.799+0000 7fa5c533a640 1 --2- 172.21.15.102:0/2589051552 >> v2:172.21.15.102:3300/0 conn(0x7fa5c0128330 0x7fa5c0128700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:04.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.799+0000 7fa5c533a640 1 -- 172.21.15.102:0/2589051552 >> v1:172.21.15.102:6789/0 conn(0x7fa5c0128c40 legacy=0x7fa5c012b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:04.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.799+0000 7fa5c533a640 1 -- 172.21.15.102:0/2589051552 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c00985d0 con 0x7fa5c012bf50 2024-09-13T17:44:04.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.799+0000 7fa5c4b39640 1 --2- 172.21.15.102:0/2589051552 >> v2:172.21.15.102:3300/0 conn(0x7fa5c0128330 0x7fa5c0128700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:04.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.800+0000 7fa5c533a640 1 --2- 172.21.15.102:0/2589051552 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa5c012bf50 0x7fa5c0133b90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8009e60 tx=0x7fa5b8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c2fcad91a9e4df0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:04.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.800+0000 7fa5bf7fe640 1 -- 172.21.15.102:0/2589051552 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8007550 con 0x7fa5c012bf50 2024-09-13T17:44:04.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.800+0000 7fa5bf7fe640 1 -- 172.21.15.102:0/2589051552 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b802d050 con 0x7fa5c012bf50 2024-09-13T17:44:04.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.800+0000 7fa5bf7fe640 1 -- 172.21.15.102:0/2589051552 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8007920 con 0x7fa5c012bf50 2024-09-13T17:44:04.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.800+0000 7fa5c6d9d640 1 -- 172.21.15.102:0/2589051552 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa5c012bf50 msgr2=0x7fa5c0133b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:04.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.800+0000 7fa5c6d9d640 1 --2- 172.21.15.102:0/2589051552 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa5c012bf50 0x7fa5c0133b90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8009e60 tx=0x7fa5b8009780 comp rx=0 tx=0).stop 2024-09-13T17:44:04.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.800+0000 7fa5c6d9d640 1 -- 172.21.15.102:0/2589051552 shutdown_connections 2024-09-13T17:44:04.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.800+0000 7fa5c6d9d640 1 --2- 172.21.15.102:0/2589051552 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa5c012bf50 0x7fa5c0133b90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:04.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.800+0000 7fa5c6d9d640 1 --2- 172.21.15.102:0/2589051552 >> v2:172.21.15.102:3300/0 conn(0x7fa5c0128330 0x7fa5c0128700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:04.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.800+0000 7fa5c6d9d640 1 -- 172.21.15.102:0/2589051552 >> 172.21.15.102:0/2589051552 conn(0x7fa5c009e820 msgr2=0x7fa5c009ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:04.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.800+0000 7fa5c6d9d640 1 -- 172.21.15.102:0/2589051552 shutdown_connections 2024-09-13T17:44:04.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.800+0000 7fa5c6d9d640 1 -- 172.21.15.102:0/2589051552 wait complete. 2024-09-13T17:44:04.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.801+0000 7fa5c6d9d640 1 Processor -- start 2024-09-13T17:44:04.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.801+0000 7fa5c6d9d640 1 -- start start 2024-09-13T17:44:04.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.802+0000 7fa5c6d9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5c0128330 0x7fa5c012ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:04.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.802+0000 7fa5c6d9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa5c0128c40 0x7fa5c006d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:04.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.802+0000 7fa5c6d9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa5c012bf50 0x7fa5c006e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:04.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.802+0000 7fa5c6d9d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c012e120 con 0x7fa5c0128330 2024-09-13T17:44:04.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.802+0000 7fa5c6d9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c012e6b0 con 0x7fa5c0128c40 2024-09-13T17:44:04.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.802+0000 7fa5c6d9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c006dd90 con 0x7fa5c012bf50 2024-09-13T17:44:04.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.802+0000 7fa5bffff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa5c0128c40 0x7fa5c006d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:04.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.802+0000 7fa5c4b39640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5c0128330 0x7fa5c012ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:04.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.802+0000 7fa5c4b39640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5c0128330 0x7fa5c012ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34328/0 (socket says 172.21.15.102:34328) 2024-09-13T17:44:04.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.802+0000 7fa5bffff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa5c0128c40 0x7fa5c006d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:46948/0 (socket says 172.21.15.102:46948) 2024-09-13T17:44:04.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.802+0000 7fa5c4b39640 1 -- 172.21.15.102:0/3901660570 learned_addr learned my addr 172.21.15.102:0/3901660570 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:04.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.802+0000 7fa5c533a640 1 --2- 172.21.15.102:0/3901660570 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa5c012bf50 0x7fa5c006e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:04.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.802+0000 7fa5c4b39640 1 -- 172.21.15.102:0/3901660570 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa5c012bf50 msgr2=0x7fa5c006e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:04.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.802+0000 7fa5c4b39640 1 --2- 172.21.15.102:0/3901660570 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa5c012bf50 0x7fa5c006e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:04.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.802+0000 7fa5c4b39640 1 -- 172.21.15.102:0/3901660570 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa5c0128c40 msgr2=0x7fa5c006d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:04.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.802+0000 7fa5c4b39640 1 --2- 172.21.15.102:0/3901660570 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa5c0128c40 0x7fa5c006d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:04.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.802+0000 7fa5c4b39640 1 -- 172.21.15.102:0/3901660570 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b4009390 con 0x7fa5c0128330 2024-09-13T17:44:04.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.802+0000 7fa5bffff640 1 --2- 172.21.15.102:0/3901660570 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa5c0128c40 0x7fa5c006d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:04.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.802+0000 7fa5c4b39640 1 --2- 172.21.15.102:0/3901660570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5c0128330 0x7fa5c012ee90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400ec80 tx=0x7fa5b40090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:04.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.802+0000 7fa5bcff9640 1 -- 172.21.15.102:0/3901660570 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b400eec0 con 0x7fa5c0128330 2024-09-13T17:44:04.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.802+0000 7fa5bcff9640 1 -- 172.21.15.102:0/3901660570 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b400dbd0 con 0x7fa5c0128330 2024-09-13T17:44:04.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.802+0000 7fa5c6d9d640 1 -- 172.21.15.102:0/3901660570 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b8009a20 con 0x7fa5c0128330 2024-09-13T17:44:04.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.802+0000 7fa5bcff9640 1 -- 172.21.15.102:0/3901660570 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4010440 con 0x7fa5c0128330 2024-09-13T17:44:04.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.802+0000 7fa5c6d9d640 1 -- 172.21.15.102:0/3901660570 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c006e8b0 con 0x7fa5c0128330 2024-09-13T17:44:04.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.803+0000 7fa5bcff9640 1 -- 172.21.15.102:0/3901660570 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fa5b40107b0 con 0x7fa5c0128330 2024-09-13T17:44:04.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.803+0000 7fa5bcff9640 1 -- 172.21.15.102:0/3901660570 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 2808+0+0 (secure 0 0 0) 0x7fa5b4014070 con 0x7fa5c0128330 2024-09-13T17:44:04.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.803+0000 7fa5c6d9d640 1 -- 172.21.15.102:0/3901660570 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c006eb00 con 0x7fa5c0128330 2024-09-13T17:44:04.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:04.808+0000 7fa5bcff9640 1 -- 172.21.15.102:0/3901660570 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fa5c006eb00 con 0x7fa5c0128330 2024-09-13T17:44:05.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.016+0000 7fa5c6d9d640 1 -- 172.21.15.102:0/3901660570 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fa5c012cb30 con 0x7fa5c0128330 2024-09-13T17:44:05.021 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.020+0000 7fa5bcff9640 1 -- 172.21.15.102:0/3901660570 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa5c012cb30 con 0x7fa5c0128330 2024-09-13T17:44:05.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.021+0000 7fa5c6d9d640 1 -- 172.21.15.102:0/3901660570 >> v1:172.21.15.102:6816/34937 conn(0x7fa5a4031160 legacy=0x7fa5a4033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:05.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.021+0000 7fa5c6d9d640 1 -- 172.21.15.102:0/3901660570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5c0128330 msgr2=0x7fa5c012ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:05.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.021+0000 7fa5c6d9d640 1 --2- 172.21.15.102:0/3901660570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5c0128330 0x7fa5c012ee90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400ec80 tx=0x7fa5b40090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:05.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.022+0000 7fa5c6d9d640 1 -- 172.21.15.102:0/3901660570 shutdown_connections 2024-09-13T17:44:05.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.022+0000 7fa5c6d9d640 1 --2- 172.21.15.102:0/3901660570 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa5c012bf50 0x7fa5c006e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:05.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.022+0000 7fa5c6d9d640 1 --2- 172.21.15.102:0/3901660570 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa5c0128c40 0x7fa5c006d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:05.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.022+0000 7fa5c6d9d640 1 --2- 172.21.15.102:0/3901660570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5c0128330 0x7fa5c012ee90 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:05.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.022+0000 7fa5c6d9d640 1 -- 172.21.15.102:0/3901660570 >> 172.21.15.102:0/3901660570 conn(0x7fa5c009e820 msgr2=0x7fa5c009c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:05.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.022+0000 7fa5c6d9d640 1 -- 172.21.15.102:0/3901660570 shutdown_connections 2024-09-13T17:44:05.024 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.022+0000 7fa5c6d9d640 1 -- 172.21.15.102:0/3901660570 wait complete. 2024-09-13T17:44:05.034 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-13T17:44:05.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.126+0000 7f12d175d640 1 Processor -- start 2024-09-13T17:44:05.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.126+0000 7f12d175d640 1 -- start start 2024-09-13T17:44:05.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.127+0000 7f12d175d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f12cc068920 0x7f12cc09d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:05.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.127+0000 7f12d175d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f12cc09e020 0x7f12cc12fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:05.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.127+0000 7f12d175d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12cc0695a0 con 0x7f12cc09dc50 2024-09-13T17:44:05.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.127+0000 7f12d175d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12cc068cf0 con 0x7f12cc09e020 2024-09-13T17:44:05.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.127+0000 7f12d175d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc09e4a0 con 0x7f12cc068920 2024-09-13T17:44:05.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.127+0000 7f12caffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f12cc068920 0x7f12cc09d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:05.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.127+0000 7f12cb7fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f12cc09e020 0x7f12cc12fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:05.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.127+0000 7f12ca7fc640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f12cc09dc50 0x7f12cc12f310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:37412/0 (socket says 172.21.15.102:37412) 2024-09-13T17:44:05.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.127+0000 7f12ca7fc640 1 -- 172.21.15.102:0/2323523775 learned_addr learned my addr 172.21.15.102:0/2323523775 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:44:05.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.127+0000 7f12caffd640 1 -- 172.21.15.102:0/2323523775 >> v2:172.21.15.102:3300/0 conn(0x7f12cc09e020 msgr2=0x7f12cc12fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:05.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.127+0000 7f12caffd640 1 --2- 172.21.15.102:0/2323523775 >> v2:172.21.15.102:3300/0 conn(0x7f12cc09e020 0x7f12cc12fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:05.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.127+0000 7f12caffd640 1 -- 172.21.15.102:0/2323523775 >> v1:172.21.15.102:6789/0 conn(0x7f12cc09dc50 legacy=0x7f12cc12f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:05.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.127+0000 7f12caffd640 1 -- 172.21.15.102:0/2323523775 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc09f090 con 0x7f12cc068920 2024-09-13T17:44:05.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.127+0000 7f12cb7fe640 1 --2- 172.21.15.102:0/2323523775 >> v2:172.21.15.102:3300/0 conn(0x7f12cc09e020 0x7f12cc12fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:05.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.128+0000 7f12caffd640 1 --2- 172.21.15.102:0/2323523775 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f12cc068920 0x7f12cc09d710 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009850 tx=0x7f12bc0090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c3525af99a7bc87 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:05.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.128+0000 7f12c9ffb640 1 -- 172.21.15.102:0/2323523775 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc02ac80 con 0x7f12cc068920 2024-09-13T17:44:05.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.128+0000 7f12c9ffb640 1 -- 172.21.15.102:0/2323523775 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12bc02ade0 con 0x7f12cc068920 2024-09-13T17:44:05.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.128+0000 7f12c9ffb640 1 -- 172.21.15.102:0/2323523775 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc032670 con 0x7f12cc068920 2024-09-13T17:44:05.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.129+0000 7f12d175d640 1 -- 172.21.15.102:0/2323523775 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f12cc068920 msgr2=0x7f12cc09d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:05.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.129+0000 7f12d175d640 1 --2- 172.21.15.102:0/2323523775 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f12cc068920 0x7f12cc09d710 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009850 tx=0x7f12bc0090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:05.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.129+0000 7f12d175d640 1 -- 172.21.15.102:0/2323523775 shutdown_connections 2024-09-13T17:44:05.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.129+0000 7f12d175d640 1 --2- 172.21.15.102:0/2323523775 >> v2:172.21.15.102:3300/0 conn(0x7f12cc09e020 0x7f12cc12fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:05.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.129+0000 7f12d175d640 1 --2- 172.21.15.102:0/2323523775 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f12cc068920 0x7f12cc09d710 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:05.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.129+0000 7f12d175d640 1 -- 172.21.15.102:0/2323523775 >> 172.21.15.102:0/2323523775 conn(0x7f12cc06f320 msgr2=0x7f12cc0a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:05.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.129+0000 7f12d175d640 1 -- 172.21.15.102:0/2323523775 shutdown_connections 2024-09-13T17:44:05.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.129+0000 7f12d175d640 1 -- 172.21.15.102:0/2323523775 wait complete. 2024-09-13T17:44:05.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.130+0000 7f12d175d640 1 Processor -- start 2024-09-13T17:44:05.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.130+0000 7f12d175d640 1 -- start start 2024-09-13T17:44:05.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.130+0000 7f12d175d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12cc068920 0x7f12cc121bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:05.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.130+0000 7f12d175d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f12cc09dc50 0x7f12cc11dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:05.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.130+0000 7f12d175d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f12cc09e020 0x7f12cc11e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:05.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.130+0000 7f12d175d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc09e620 con 0x7f12cc068920 2024-09-13T17:44:05.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.130+0000 7f12d175d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc121110 con 0x7f12cc09e020 2024-09-13T17:44:05.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.130+0000 7f12d175d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc1216e0 con 0x7f12cc09dc50 2024-09-13T17:44:05.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.130+0000 7f12caffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12cc068920 0x7f12cc121bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:05.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.130+0000 7f12ca7fc640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f12cc09dc50 0x7f12cc11dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:05.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.130+0000 7f12caffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12cc068920 0x7f12cc121bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34348/0 (socket says 172.21.15.102:34348) 2024-09-13T17:44:05.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.130+0000 7f12ca7fc640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f12cc09dc50 0x7f12cc11dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54260/0 (socket says 172.21.15.102:54260) 2024-09-13T17:44:05.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.130+0000 7f12caffd640 1 -- 172.21.15.102:0/3338134001 learned_addr learned my addr 172.21.15.102:0/3338134001 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:05.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.130+0000 7f12cb7fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f12cc09e020 0x7f12cc11e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:05.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.130+0000 7f12caffd640 1 -- 172.21.15.102:0/3338134001 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f12cc09dc50 msgr2=0x7f12cc11dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:05.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.130+0000 7f12caffd640 1 --2- 172.21.15.102:0/3338134001 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f12cc09dc50 0x7f12cc11dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:05.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.130+0000 7f12caffd640 1 -- 172.21.15.102:0/3338134001 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f12cc09e020 msgr2=0x7f12cc11e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:05.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.130+0000 7f12caffd640 1 --2- 172.21.15.102:0/3338134001 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f12cc09e020 0x7f12cc11e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:05.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.131+0000 7f12caffd640 1 -- 172.21.15.102:0/3338134001 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c4009a20 con 0x7f12cc068920 2024-09-13T17:44:05.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.131+0000 7f12caffd640 1 --2- 172.21.15.102:0/3338134001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12cc068920 0x7f12cc121bb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f12bc02a5b0 tx=0x7f12bc032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:05.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.131+0000 7f12b2ffd640 1 -- 172.21.15.102:0/3338134001 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc03d700 con 0x7f12cc068920 2024-09-13T17:44:05.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.131+0000 7f12b2ffd640 1 -- 172.21.15.102:0/3338134001 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12bc03d860 con 0x7f12cc068920 2024-09-13T17:44:05.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.131+0000 7f12d175d640 1 -- 172.21.15.102:0/3338134001 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12bc009390 con 0x7f12cc068920 2024-09-13T17:44:05.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.131+0000 7f12b2ffd640 1 -- 172.21.15.102:0/3338134001 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc03dc00 con 0x7f12cc068920 2024-09-13T17:44:05.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.131+0000 7f12d175d640 1 -- 172.21.15.102:0/3338134001 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12cc11f8b0 con 0x7f12cc068920 2024-09-13T17:44:05.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.132+0000 7f12b2ffd640 1 -- 172.21.15.102:0/3338134001 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f12bc039040 con 0x7f12cc068920 2024-09-13T17:44:05.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.132+0000 7f12b2ffd640 1 -- 172.21.15.102:0/3338134001 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 2808+0+0 (secure 0 0 0) 0x7f12bc061800 con 0x7f12cc068920 2024-09-13T17:44:05.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.132+0000 7f12d175d640 1 -- 172.21.15.102:0/3338134001 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12cc0a2360 con 0x7f12cc068920 2024-09-13T17:44:05.138 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.136+0000 7f12b2ffd640 1 -- 172.21.15.102:0/3338134001 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f12bc042070 con 0x7f12cc068920 2024-09-13T17:44:05.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.346+0000 7f12d175d640 1 -- 172.21.15.102:0/3338134001 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f12cc121330 con 0x7f12cc068920 2024-09-13T17:44:05.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.349+0000 7f12b2ffd640 1 -- 172.21.15.102:0/3338134001 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f12bc037030 con 0x7f12cc068920 2024-09-13T17:44:05.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.351+0000 7f12d175d640 1 -- 172.21.15.102:0/3338134001 >> v1:172.21.15.102:6816/34937 conn(0x7f12b8031190 legacy=0x7f12b8033640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:05.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.351+0000 7f12d175d640 1 -- 172.21.15.102:0/3338134001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12cc068920 msgr2=0x7f12cc121bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:05.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.351+0000 7f12d175d640 1 --2- 172.21.15.102:0/3338134001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12cc068920 0x7f12cc121bb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f12bc02a5b0 tx=0x7f12bc032000 comp rx=0 tx=0).stop 2024-09-13T17:44:05.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.351+0000 7f12d175d640 1 -- 172.21.15.102:0/3338134001 shutdown_connections 2024-09-13T17:44:05.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.351+0000 7f12d175d640 1 --2- 172.21.15.102:0/3338134001 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f12cc09e020 0x7f12cc11e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:05.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.351+0000 7f12d175d640 1 --2- 172.21.15.102:0/3338134001 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f12cc09dc50 0x7f12cc11dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:05.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.351+0000 7f12d175d640 1 --2- 172.21.15.102:0/3338134001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12cc068920 0x7f12cc121bb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:05.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.351+0000 7f12d175d640 1 -- 172.21.15.102:0/3338134001 >> 172.21.15.102:0/3338134001 conn(0x7f12cc06f320 msgr2=0x7f12cc09b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:05.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.351+0000 7f12d175d640 1 -- 172.21.15.102:0/3338134001 shutdown_connections 2024-09-13T17:44:05.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.351+0000 7f12d175d640 1 -- 172.21.15.102:0/3338134001 wait complete. 2024-09-13T17:44:05.365 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:44:05.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.456+0000 7f67dcf9d640 1 Processor -- start 2024-09-13T17:44:05.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.456+0000 7f67dcf9d640 1 -- start start 2024-09-13T17:44:05.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.456+0000 7f67dcf9d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f67d8128e10 0x7f67d8130b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:05.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.456+0000 7f67dcf9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67d81310b0 0x7f67d8133490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:05.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.456+0000 7f67dcf9d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67d8097be0 con 0x7f67d8128330 2024-09-13T17:44:05.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.456+0000 7f67dcf9d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67d8129200 con 0x7f67d8128e10 2024-09-13T17:44:05.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.456+0000 7f67dcf9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d8098e70 con 0x7f67d81310b0 2024-09-13T17:44:05.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.456+0000 7f67d5d9b640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f67d8128e10 0x7f67d8130b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:05.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.456+0000 7f67d6d9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67d81310b0 0x7f67d8133490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:05.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.456+0000 7f67d5d9b640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f67d8128e10 0x7f67d8130b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34360/0 (socket says 172.21.15.102:34360) 2024-09-13T17:44:05.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.456+0000 7f67d5d9b640 1 -- 172.21.15.102:0/1304303545 learned_addr learned my addr 172.21.15.102:0/1304303545 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:05.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.456+0000 7f67d6d9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67d81310b0 0x7f67d8133490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:46972/0 (socket says 172.21.15.102:46972) 2024-09-13T17:44:05.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.456+0000 7f67d659c640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f67d8128330 0x7f67d8128700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:37424/0 (socket says 172.21.15.102:37424) 2024-09-13T17:44:05.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.457+0000 7f67d6d9d640 1 -- 172.21.15.102:0/1304303545 >> v2:172.21.15.102:3300/0 conn(0x7f67d8128e10 msgr2=0x7f67d8130b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:05.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.457+0000 7f67d6d9d640 1 --2- 172.21.15.102:0/1304303545 >> v2:172.21.15.102:3300/0 conn(0x7f67d8128e10 0x7f67d8130b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:05.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.457+0000 7f67d6d9d640 1 -- 172.21.15.102:0/1304303545 >> v1:172.21.15.102:6789/0 conn(0x7f67d8128330 legacy=0x7f67d8128700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:05.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.457+0000 7f67d6d9d640 1 -- 172.21.15.102:0/1304303545 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d8098310 con 0x7f67d81310b0 2024-09-13T17:44:05.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.457+0000 7f67d5d9b640 1 --2- 172.21.15.102:0/1304303545 >> v2:172.21.15.102:3300/0 conn(0x7f67d8128e10 0x7f67d8130b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:05.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.457+0000 7f67d6d9d640 1 --2- 172.21.15.102:0/1304303545 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67d81310b0 0x7f67d8133490 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f67d0009e90 tx=0x7f67d0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5274735d6ceb327 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:05.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.458+0000 7f67d559a640 1 -- 172.21.15.102:0/1304303545 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d0032620 con 0x7f67d81310b0 2024-09-13T17:44:05.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.458+0000 7f67d559a640 1 -- 172.21.15.102:0/1304303545 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d002e040 con 0x7f67d81310b0 2024-09-13T17:44:05.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.458+0000 7f67d559a640 1 -- 172.21.15.102:0/1304303545 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d00329f0 con 0x7f67d81310b0 2024-09-13T17:44:05.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.458+0000 7f67dcf9d640 1 -- 172.21.15.102:0/1304303545 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67d81310b0 msgr2=0x7f67d8133490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:05.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.458+0000 7f67dcf9d640 1 --2- 172.21.15.102:0/1304303545 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67d81310b0 0x7f67d8133490 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f67d0009e90 tx=0x7f67d0009780 comp rx=0 tx=0).stop 2024-09-13T17:44:05.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.458+0000 7f67dcf9d640 1 -- 172.21.15.102:0/1304303545 shutdown_connections 2024-09-13T17:44:05.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.458+0000 7f67dcf9d640 1 --2- 172.21.15.102:0/1304303545 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67d81310b0 0x7f67d8133490 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:05.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.458+0000 7f67dcf9d640 1 --2- 172.21.15.102:0/1304303545 >> v2:172.21.15.102:3300/0 conn(0x7f67d8128e10 0x7f67d8130b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:05.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.458+0000 7f67dcf9d640 1 -- 172.21.15.102:0/1304303545 >> 172.21.15.102:0/1304303545 conn(0x7f67d809e820 msgr2=0x7f67d809ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:05.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.458+0000 7f67dcf9d640 1 -- 172.21.15.102:0/1304303545 shutdown_connections 2024-09-13T17:44:05.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.458+0000 7f67dcf9d640 1 -- 172.21.15.102:0/1304303545 wait complete. 2024-09-13T17:44:05.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.459+0000 7f67dcf9d640 1 Processor -- start 2024-09-13T17:44:05.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.459+0000 7f67dcf9d640 1 -- start start 2024-09-13T17:44:05.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.459+0000 7f67dcf9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f67d8128330 0x7f67d812f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:05.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.459+0000 7f67dcf9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67d8128e10 0x7f67d812f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:05.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.459+0000 7f67dcf9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67d81310b0 0x7f67d812fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:05.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.459+0000 7f67dcf9d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d812e190 con 0x7f67d81310b0 2024-09-13T17:44:05.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.459+0000 7f67dcf9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d8130e50 con 0x7f67d8128e10 2024-09-13T17:44:05.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.459+0000 7f67dcf9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67d812ded0 con 0x7f67d8128330 2024-09-13T17:44:05.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.459+0000 7f67d5d9b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67d8128e10 0x7f67d812f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.459+0000 7f67d6d9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67d81310b0 0x7f67d812fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.459+0000 7f67d659c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f67d8128330 0x7f67d812f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.459+0000 7f67d5d9b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67d8128e10 0x7f67d812f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:46982/0 (socket says 172.21.15.102:46982) 2024-09-13T17:44:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.459+0000 7f67d5d9b640 1 -- 172.21.15.102:0/4204359845 learned_addr learned my addr 172.21.15.102:0/4204359845 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.460+0000 7f67d5d9b640 1 -- 172.21.15.102:0/4204359845 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f67d8128330 msgr2=0x7f67d812f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.460+0000 7f67d5d9b640 1 --2- 172.21.15.102:0/4204359845 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f67d8128330 0x7f67d812f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.460+0000 7f67d5d9b640 1 -- 172.21.15.102:0/4204359845 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67d81310b0 msgr2=0x7f67d812fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.460+0000 7f67d5d9b640 1 --2- 172.21.15.102:0/4204359845 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67d81310b0 0x7f67d812fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.460+0000 7f67d5d9b640 1 -- 172.21.15.102:0/4204359845 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67cc009450 con 0x7f67d8128e10 2024-09-13T17:44:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.460+0000 7f67d6d9d640 1 --2- 172.21.15.102:0/4204359845 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67d81310b0 0x7f67d812fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.460+0000 7f67d5d9b640 1 --2- 172.21.15.102:0/4204359845 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67d8128e10 0x7f67d812f590 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f67cc00ec80 tx=0x7f67cc0091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.460+0000 7f67c67fc640 1 -- 172.21.15.102:0/4204359845 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67cc00eec0 con 0x7f67d8128e10 2024-09-13T17:44:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.460+0000 7f67c67fc640 1 -- 172.21.15.102:0/4204359845 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67cc00cbc0 con 0x7f67d8128e10 2024-09-13T17:44:05.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.460+0000 7f67dcf9d640 1 -- 172.21.15.102:0/4204359845 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d0009a20 con 0x7f67d8128e10 2024-09-13T17:44:05.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.460+0000 7f67c67fc640 1 -- 172.21.15.102:0/4204359845 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67cc013470 con 0x7f67d8128e10 2024-09-13T17:44:05.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.460+0000 7f67dcf9d640 1 -- 172.21.15.102:0/4204359845 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d812c560 con 0x7f67d8128e10 2024-09-13T17:44:05.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.461+0000 7f67c67fc640 1 -- 172.21.15.102:0/4204359845 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f67cc013820 con 0x7f67d8128e10 2024-09-13T17:44:05.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.461+0000 7f67dcf9d640 1 -- 172.21.15.102:0/4204359845 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d8130120 con 0x7f67d8128e10 2024-09-13T17:44:05.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.461+0000 7f67c67fc640 1 -- 172.21.15.102:0/4204359845 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 2808+0+0 (secure 0 0 0) 0x7f67cc042110 con 0x7f67d8128e10 2024-09-13T17:44:05.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.465+0000 7f67c67fc640 1 -- 172.21.15.102:0/4204359845 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f67cc021070 con 0x7f67d8128e10 2024-09-13T17:44:05.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.619+0000 7f67dcf9d640 1 -- 172.21.15.102:0/4204359845 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67d8064000 con 0x7f67d8128e10 2024-09-13T17:44:05.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.620+0000 7f67c67fc640 1 -- 172.21.15.102:0/4204359845 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+8114 (secure 0 0 0) 0x7f67cc0236b0 con 0x7f67d8128e10 2024-09-13T17:44:05.622 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:44:05.622 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":28,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:44:03.036728+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:44:05.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.621+0000 7f67dcf9d640 1 -- 172.21.15.102:0/4204359845 >> v1:172.21.15.102:6816/34937 conn(0x7f67b8031160 legacy=0x7f67b8033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:05.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.621+0000 7f67dcf9d640 1 -- 172.21.15.102:0/4204359845 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67d8128e10 msgr2=0x7f67d812f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:05.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.621+0000 7f67dcf9d640 1 --2- 172.21.15.102:0/4204359845 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67d8128e10 0x7f67d812f590 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f67cc00ec80 tx=0x7f67cc0091d0 comp rx=0 tx=0).stop 2024-09-13T17:44:05.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.621+0000 7f67dcf9d640 1 -- 172.21.15.102:0/4204359845 shutdown_connections 2024-09-13T17:44:05.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.621+0000 7f67dcf9d640 1 --2- 172.21.15.102:0/4204359845 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67d81310b0 0x7f67d812fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:05.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.621+0000 7f67dcf9d640 1 --2- 172.21.15.102:0/4204359845 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67d8128e10 0x7f67d812f590 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:05.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.621+0000 7f67dcf9d640 1 --2- 172.21.15.102:0/4204359845 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f67d8128330 0x7f67d812f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:05.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.621+0000 7f67dcf9d640 1 -- 172.21.15.102:0/4204359845 >> 172.21.15.102:0/4204359845 conn(0x7f67d809e820 msgr2=0x7f67d8069ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:05.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.622+0000 7f67dcf9d640 1 -- 172.21.15.102:0/4204359845 shutdown_connections 2024-09-13T17:44:05.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.622+0000 7f67dcf9d640 1 -- 172.21.15.102:0/4204359845 wait complete. 2024-09-13T17:44:05.634 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:54.442612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T17:41:59.076782+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 16, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2024-09-13T17:44:05.634 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:44:05.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.728+0000 7fcaad19d640 1 Processor -- start 2024-09-13T17:44:05.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.728+0000 7fcaad19d640 1 -- start start 2024-09-13T17:44:05.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.728+0000 7fcaad19d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcaa8096a70 0x7fcaa80987c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:05.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.728+0000 7fcaad19d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcaa8099110 0x7fcaa812c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:05.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.728+0000 7fcaad19d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaa80999e0 con 0x7fcaa8098d00 2024-09-13T17:44:05.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.728+0000 7fcaad19d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa8069c30 con 0x7fcaa8099110 2024-09-13T17:44:05.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.729+0000 7fcaad19d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa806ed00 con 0x7fcaa8096a70 2024-09-13T17:44:05.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.729+0000 7fcaa6d9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcaa8096a70 0x7fcaa80987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:05.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.729+0000 7fcaa759e640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcaa8099110 0x7fcaa812c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:05.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.729+0000 7fca9e59c640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fcaa8098d00 0x7fcaa812c2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:37434/0 (socket says 172.21.15.102:37434) 2024-09-13T17:44:05.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.729+0000 7fca9e59c640 1 -- 172.21.15.102:0/3467959073 learned_addr learned my addr 172.21.15.102:0/3467959073 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:44:05.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.729+0000 7fcaa6d9d640 1 -- 172.21.15.102:0/3467959073 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcaa8099110 msgr2=0x7fcaa812c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:05.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.729+0000 7fcaa6d9d640 1 --2- 172.21.15.102:0/3467959073 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcaa8099110 0x7fcaa812c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:05.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.729+0000 7fcaa659c640 1 -- 172.21.15.102:0/3467959073 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4099173541 0 0) 0x7fcaa80999e0 con 0x7fcaa8098d00 2024-09-13T17:44:05.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.729+0000 7fcaa6d9d640 1 -- 172.21.15.102:0/3467959073 >> v1:172.21.15.102:6789/0 conn(0x7fcaa8098d00 legacy=0x7fcaa812c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:05.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.729+0000 7fcaa6d9d640 1 -- 172.21.15.102:0/3467959073 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa80997a0 con 0x7fcaa8096a70 2024-09-13T17:44:05.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.729+0000 7fcaa759e640 1 --2- 172.21.15.102:0/3467959073 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcaa8099110 0x7fcaa812c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:05.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.730+0000 7fcaa6d9d640 1 --2- 172.21.15.102:0/3467959073 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcaa8096a70 0x7fcaa80987c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fca98009880 tx=0x7fca980090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6355a315896e36f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:05.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.730+0000 7fcaa659c640 1 -- 172.21.15.102:0/3467959073 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9802acd0 con 0x7fcaa8096a70 2024-09-13T17:44:05.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.730+0000 7fcaa659c640 1 -- 172.21.15.102:0/3467959073 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9802ae30 con 0x7fcaa8096a70 2024-09-13T17:44:05.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.730+0000 7fcaa659c640 1 -- 172.21.15.102:0/3467959073 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca98032670 con 0x7fcaa8096a70 2024-09-13T17:44:05.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.730+0000 7fcaad19d640 1 -- 172.21.15.102:0/3467959073 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcaa8096a70 msgr2=0x7fcaa80987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:05.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.730+0000 7fcaad19d640 1 --2- 172.21.15.102:0/3467959073 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcaa8096a70 0x7fcaa80987c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fca98009880 tx=0x7fca980090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:05.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.730+0000 7fcaad19d640 1 -- 172.21.15.102:0/3467959073 shutdown_connections 2024-09-13T17:44:05.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.730+0000 7fcaad19d640 1 --2- 172.21.15.102:0/3467959073 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcaa8099110 0x7fcaa812c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:05.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.730+0000 7fcaad19d640 1 --2- 172.21.15.102:0/3467959073 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcaa8096a70 0x7fcaa80987c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:05.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.730+0000 7fcaad19d640 1 -- 172.21.15.102:0/3467959073 >> 172.21.15.102:0/3467959073 conn(0x7fcaa809e720 msgr2=0x7fcaa809eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:05.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.730+0000 7fcaad19d640 1 -- 172.21.15.102:0/3467959073 shutdown_connections 2024-09-13T17:44:05.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.730+0000 7fcaad19d640 1 -- 172.21.15.102:0/3467959073 wait complete. 2024-09-13T17:44:05.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.731+0000 7fcaad19d640 1 Processor -- start 2024-09-13T17:44:05.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.731+0000 7fcaad19d640 1 -- start start 2024-09-13T17:44:05.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.732+0000 7fcaad19d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcaa8096a70 0x7fcaa81177b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:05.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.732+0000 7fcaad19d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcaa8098d00 0x7fcaa8117cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:05.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.732+0000 7fcaad19d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcaa8099110 0x7fcaa813c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:05.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.732+0000 7fcaad19d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa812b820 con 0x7fcaa8098d00 2024-09-13T17:44:05.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.732+0000 7fcaad19d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa809a410 con 0x7fcaa8099110 2024-09-13T17:44:05.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.732+0000 7fcaad19d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa81198a0 con 0x7fcaa8096a70 2024-09-13T17:44:05.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.732+0000 7fcaa6d9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcaa8096a70 0x7fcaa81177b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:05.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.732+0000 7fcaa6d9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcaa8096a70 0x7fcaa81177b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54270/0 (socket says 172.21.15.102:54270) 2024-09-13T17:44:05.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.732+0000 7fcaa759e640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcaa8099110 0x7fcaa813c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:05.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.732+0000 7fcaa6d9d640 1 -- 172.21.15.102:0/958932607 learned_addr learned my addr 172.21.15.102:0/958932607 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:05.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.732+0000 7fca9e59c640 1 --2- 172.21.15.102:0/958932607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcaa8098d00 0x7fcaa8117cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:05.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.732+0000 7fcaa759e640 1 -- 172.21.15.102:0/958932607 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcaa8096a70 msgr2=0x7fcaa81177b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:05.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.732+0000 7fcaa759e640 1 --2- 172.21.15.102:0/958932607 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcaa8096a70 0x7fcaa81177b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:05.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.732+0000 7fcaa759e640 1 -- 172.21.15.102:0/958932607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcaa8098d00 msgr2=0x7fcaa8117cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:05.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.732+0000 7fcaa759e640 1 --2- 172.21.15.102:0/958932607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcaa8098d00 0x7fcaa8117cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:05.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.732+0000 7fcaa759e640 1 -- 172.21.15.102:0/958932607 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa0009a20 con 0x7fcaa8099110 2024-09-13T17:44:05.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.732+0000 7fcaa6d9d640 1 --2- 172.21.15.102:0/958932607 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcaa8096a70 0x7fcaa81177b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:05.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.732+0000 7fcaa759e640 1 --2- 172.21.15.102:0/958932607 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcaa8099110 0x7fcaa813c6d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0009f40 tx=0x7fcaa000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:05.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.732+0000 7fca9f7fe640 1 -- 172.21.15.102:0/958932607 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa000ce10 con 0x7fcaa8099110 2024-09-13T17:44:05.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.732+0000 7fca9f7fe640 1 -- 172.21.15.102:0/958932607 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa000f040 con 0x7fcaa8099110 2024-09-13T17:44:05.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.732+0000 7fcaad19d640 1 -- 172.21.15.102:0/958932607 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca98009390 con 0x7fcaa8099110 2024-09-13T17:44:05.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.732+0000 7fca9f7fe640 1 -- 172.21.15.102:0/958932607 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa00146d0 con 0x7fcaa8099110 2024-09-13T17:44:05.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.732+0000 7fcaad19d640 1 -- 172.21.15.102:0/958932607 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa8118230 con 0x7fcaa8099110 2024-09-13T17:44:05.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.733+0000 7fca9f7fe640 1 -- 172.21.15.102:0/958932607 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fcaa0003020 con 0x7fcaa8099110 2024-09-13T17:44:05.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.733+0000 7fcaad19d640 1 -- 172.21.15.102:0/958932607 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaa8064000 con 0x7fcaa8099110 2024-09-13T17:44:05.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.733+0000 7fca9f7fe640 1 -- 172.21.15.102:0/958932607 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 2808+0+0 (secure 0 0 0) 0x7fcaa0042600 con 0x7fcaa8099110 2024-09-13T17:44:05.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.738+0000 7fca9f7fe640 1 -- 172.21.15.102:0/958932607 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fcaa0012d90 con 0x7fcaa8099110 2024-09-13T17:44:05.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.886+0000 7fcaad19d640 1 -- 172.21.15.102:0/958932607 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcaa80676b0 con 0x7fcaa8099110 2024-09-13T17:44:05.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.886+0000 7fca9f7fe640 1 -- 172.21.15.102:0/958932607 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcaa00180a0 con 0x7fcaa8099110 2024-09-13T17:44:05.888 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-09-13T17:44:05.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.888+0000 7fcaad19d640 1 -- 172.21.15.102:0/958932607 >> v1:172.21.15.102:6816/34937 conn(0x7fca88031160 legacy=0x7fca88033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:05.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.888+0000 7fcaad19d640 1 -- 172.21.15.102:0/958932607 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcaa8099110 msgr2=0x7fcaa813c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:05.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.888+0000 7fcaad19d640 1 --2- 172.21.15.102:0/958932607 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcaa8099110 0x7fcaa813c6d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0009f40 tx=0x7fcaa000b040 comp rx=0 tx=0).stop 2024-09-13T17:44:05.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.888+0000 7fcaad19d640 1 -- 172.21.15.102:0/958932607 shutdown_connections 2024-09-13T17:44:05.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.888+0000 7fcaad19d640 1 --2- 172.21.15.102:0/958932607 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fcaa8099110 0x7fcaa813c6d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:05.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.888+0000 7fcaad19d640 1 --2- 172.21.15.102:0/958932607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcaa8098d00 0x7fcaa8117cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:05.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.888+0000 7fcaad19d640 1 --2- 172.21.15.102:0/958932607 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fcaa8096a70 0x7fcaa81177b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:05.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.888+0000 7fcaad19d640 1 -- 172.21.15.102:0/958932607 >> 172.21.15.102:0/958932607 conn(0x7fcaa809e720 msgr2=0x7fcaa812f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:05.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.888+0000 7fcaad19d640 1 -- 172.21.15.102:0/958932607 shutdown_connections 2024-09-13T17:44:05.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.889+0000 7fcaad19d640 1 -- 172.21.15.102:0/958932607 wait complete. 2024-09-13T17:44:05.901 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-13T17:44:05.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.994+0000 7f0090f5d640 1 Processor -- start 2024-09-13T17:44:05.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.994+0000 7f0090f5d640 1 -- start start 2024-09-13T17:44:05.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.995+0000 7f0090f5d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f008c0689d0 0x7f008c09d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:05.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.995+0000 7f0090f5d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f008c09dd70 0x7f008c12f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:05.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.995+0000 7f0090f5d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f008c096b10 con 0x7f008c09e160 2024-09-13T17:44:05.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.995+0000 7f0090f5d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f008c096db0 con 0x7f008c0689d0 2024-09-13T17:44:05.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.995+0000 7f0090f5d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f008c097730 con 0x7f008c09dd70 2024-09-13T17:44:05.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.995+0000 7f008a59c640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f008c0689d0 0x7f008c09d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:05.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.995+0000 7f0089d9b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f008c09dd70 0x7f008c12f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:05.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.995+0000 7f008a59c640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f008c0689d0 0x7f008c09d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34370/0 (socket says 172.21.15.102:34370) 2024-09-13T17:44:05.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.995+0000 7f008a59c640 1 -- 172.21.15.102:0/1754904472 learned_addr learned my addr 172.21.15.102:0/1754904472 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:05.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.995+0000 7f0089d9b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f008c09dd70 0x7f008c12f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54274/0 (socket says 172.21.15.102:54274) 2024-09-13T17:44:05.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.995+0000 7f008ad9d640 1 --1- 172.21.15.102:0/1754904472 >> v1:172.21.15.102:6789/0 conn(0x7f008c09e160 0x7f008c12fb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:37450/0 (socket says 172.21.15.102:37450) 2024-09-13T17:44:05.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.995+0000 7f0089d9b640 1 -- 172.21.15.102:0/1754904472 >> v2:172.21.15.102:3300/0 conn(0x7f008c0689d0 msgr2=0x7f008c09d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:05.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.995+0000 7f0089d9b640 1 --2- 172.21.15.102:0/1754904472 >> v2:172.21.15.102:3300/0 conn(0x7f008c0689d0 0x7f008c09d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:05.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.995+0000 7f0089d9b640 1 -- 172.21.15.102:0/1754904472 >> v1:172.21.15.102:6789/0 conn(0x7f008c09e160 legacy=0x7f008c12fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:05.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.995+0000 7f0089d9b640 1 -- 172.21.15.102:0/1754904472 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f008c0696e0 con 0x7f008c09dd70 2024-09-13T17:44:05.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.995+0000 7f008959a640 1 -- 172.21.15.102:0/1754904472 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3845842254 0 0) 0x7f008c096b10 con 0x7f008c09e160 2024-09-13T17:44:05.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.995+0000 7f008a59c640 1 --2- 172.21.15.102:0/1754904472 >> v2:172.21.15.102:3300/0 conn(0x7f008c0689d0 0x7f008c09d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:05.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.996+0000 7f0089d9b640 1 --2- 172.21.15.102:0/1754904472 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f008c09dd70 0x7f008c12f610 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0074009850 tx=0x7f00740090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=649993c4ab03f3bc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:06.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.996+0000 7f008959a640 1 -- 172.21.15.102:0/1754904472 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007402ac80 con 0x7f008c09dd70 2024-09-13T17:44:06.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.996+0000 7f008959a640 1 -- 172.21.15.102:0/1754904472 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f007402ade0 con 0x7f008c09dd70 2024-09-13T17:44:06.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.996+0000 7f008959a640 1 -- 172.21.15.102:0/1754904472 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0074032670 con 0x7f008c09dd70 2024-09-13T17:44:06.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.997+0000 7f0090f5d640 1 -- 172.21.15.102:0/1754904472 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f008c09dd70 msgr2=0x7f008c12f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:06.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.997+0000 7f0090f5d640 1 --2- 172.21.15.102:0/1754904472 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f008c09dd70 0x7f008c12f610 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0074009850 tx=0x7f00740090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:06.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.997+0000 7f0090f5d640 1 -- 172.21.15.102:0/1754904472 shutdown_connections 2024-09-13T17:44:06.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.997+0000 7f0090f5d640 1 --2- 172.21.15.102:0/1754904472 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f008c09dd70 0x7f008c12f610 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:06.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.997+0000 7f0090f5d640 1 --2- 172.21.15.102:0/1754904472 >> v2:172.21.15.102:3300/0 conn(0x7f008c0689d0 0x7f008c09d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:06.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.997+0000 7f0090f5d640 1 -- 172.21.15.102:0/1754904472 >> 172.21.15.102:0/1754904472 conn(0x7f008c06f320 msgr2=0x7f008c0a2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:06.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.997+0000 7f0090f5d640 1 -- 172.21.15.102:0/1754904472 shutdown_connections 2024-09-13T17:44:06.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.997+0000 7f0090f5d640 1 -- 172.21.15.102:0/1754904472 wait complete. 2024-09-13T17:44:06.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.998+0000 7f0090f5d640 1 Processor -- start 2024-09-13T17:44:06.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.998+0000 7f0090f5d640 1 -- start start 2024-09-13T17:44:06.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.998+0000 7f0090f5d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f008c0689d0 0x7f008c11f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:06.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.998+0000 7f0090f5d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f008c09dd70 0x7f008c11f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:06.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.998+0000 7f0090f5d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f008c09e160 0x7f008c13e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:06.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.998+0000 7f0090f5d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008c09ebc0 con 0x7f008c09e160 2024-09-13T17:44:06.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.998+0000 7f0090f5d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f008c097aa0 con 0x7f008c0689d0 2024-09-13T17:44:06.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.998+0000 7f0090f5d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f008c121ca0 con 0x7f008c09dd70 2024-09-13T17:44:06.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.998+0000 7f008a59c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f008c0689d0 0x7f008c11f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:06.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.998+0000 7f0089d9b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f008c09dd70 0x7f008c11f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:06.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.998+0000 7f008ad9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f008c09e160 0x7f008c13e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:06.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.998+0000 7f0089d9b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f008c09dd70 0x7f008c11f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54288/0 (socket says 172.21.15.102:54288) 2024-09-13T17:44:06.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.998+0000 7f008a59c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f008c0689d0 0x7f008c11f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:47000/0 (socket says 172.21.15.102:47000) 2024-09-13T17:44:06.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.998+0000 7f0089d9b640 1 -- 172.21.15.102:0/925590036 learned_addr learned my addr 172.21.15.102:0/925590036 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:06.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.999+0000 7f0089d9b640 1 -- 172.21.15.102:0/925590036 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f008c0689d0 msgr2=0x7f008c11f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:06.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.999+0000 7f0089d9b640 1 --2- 172.21.15.102:0/925590036 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f008c0689d0 0x7f008c11f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:06.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.999+0000 7f0089d9b640 1 -- 172.21.15.102:0/925590036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f008c09e160 msgr2=0x7f008c13e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:06.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.999+0000 7f0089d9b640 1 --2- 172.21.15.102:0/925590036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f008c09e160 0x7f008c13e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:06.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.999+0000 7f0089d9b640 1 -- 172.21.15.102:0/925590036 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f007c009450 con 0x7f008c09dd70 2024-09-13T17:44:06.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.999+0000 7f008ad9d640 1 --2- 172.21.15.102:0/925590036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f008c09e160 0x7f008c13e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:06.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.999+0000 7f0089d9b640 1 --2- 172.21.15.102:0/925590036 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f008c09dd70 0x7f008c11f660 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f007402a5b0 tx=0x7f0074032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:06.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.999+0000 7f00827fc640 1 -- 172.21.15.102:0/925590036 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007403d650 con 0x7f008c09dd70 2024-09-13T17:44:06.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.999+0000 7f00827fc640 1 -- 172.21.15.102:0/925590036 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f007403d7b0 con 0x7f008c09dd70 2024-09-13T17:44:06.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.999+0000 7f0090f5d640 1 -- 172.21.15.102:0/925590036 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0074009390 con 0x7f008c09dd70 2024-09-13T17:44:06.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.999+0000 7f00827fc640 1 -- 172.21.15.102:0/925590036 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007403db30 con 0x7f008c09dd70 2024-09-13T17:44:06.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:05.999+0000 7f0090f5d640 1 -- 172.21.15.102:0/925590036 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f008c120330 con 0x7f008c09dd70 2024-09-13T17:44:06.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.000+0000 7f00827fc640 1 -- 172.21.15.102:0/925590036 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f0074039040 con 0x7f008c09dd70 2024-09-13T17:44:06.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.000+0000 7f00827fc640 1 -- 172.21.15.102:0/925590036 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 2808+0+0 (secure 0 0 0) 0x7f0074061840 con 0x7f008c09dd70 2024-09-13T17:44:06.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.000+0000 7f0090f5d640 1 -- 172.21.15.102:0/925590036 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f008c064000 con 0x7f008c09dd70 2024-09-13T17:44:06.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.004+0000 7f00827fc640 1 -- 172.21.15.102:0/925590036 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f0074037020 con 0x7f008c09dd70 2024-09-13T17:44:06.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.155+0000 7f0090f5d640 1 -- 172.21.15.102:0/925590036 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f008c0a39c0 con 0x7f008c09dd70 2024-09-13T17:44:06.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.155+0000 7f00827fc640 1 -- 172.21.15.102:0/925590036 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v28) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0074042070 con 0x7f008c09dd70 2024-09-13T17:44:06.157 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-09-13T17:44:06.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.157+0000 7f0090f5d640 1 -- 172.21.15.102:0/925590036 >> v1:172.21.15.102:6816/34937 conn(0x7f006c031160 legacy=0x7f006c033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:06.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.157+0000 7f0090f5d640 1 -- 172.21.15.102:0/925590036 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f008c09dd70 msgr2=0x7f008c11f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:06.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.157+0000 7f0090f5d640 1 --2- 172.21.15.102:0/925590036 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f008c09dd70 0x7f008c11f660 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f007402a5b0 tx=0x7f0074032000 comp rx=0 tx=0).stop 2024-09-13T17:44:06.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.157+0000 7f0090f5d640 1 -- 172.21.15.102:0/925590036 shutdown_connections 2024-09-13T17:44:06.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.157+0000 7f0090f5d640 1 --2- 172.21.15.102:0/925590036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f008c09e160 0x7f008c13e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:06.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.157+0000 7f0090f5d640 1 --2- 172.21.15.102:0/925590036 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f008c09dd70 0x7f008c11f660 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:06.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.157+0000 7f0090f5d640 1 --2- 172.21.15.102:0/925590036 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f008c0689d0 0x7f008c11f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:06.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.157+0000 7f0090f5d640 1 -- 172.21.15.102:0/925590036 >> 172.21.15.102:0/925590036 conn(0x7f008c06f320 msgr2=0x7f008c0a18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:06.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.157+0000 7f0090f5d640 1 -- 172.21.15.102:0/925590036 shutdown_connections 2024-09-13T17:44:06.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.157+0000 7f0090f5d640 1 -- 172.21.15.102:0/925590036 wait complete. 2024-09-13T17:44:06.171 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-13T17:44:06.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.262+0000 7f03aaec0640 1 Processor -- start 2024-09-13T17:44:06.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.262+0000 7f03aaec0640 1 -- start start 2024-09-13T17:44:06.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.263+0000 7f03aaec0640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f03a409de20 0x7f03a4134680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:06.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.263+0000 7f03aaec0640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f03a409e220 0x7f03a4136bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:06.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.263+0000 7f03aaec0640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03a40695a0 con 0x7f03a4068920 2024-09-13T17:44:06.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.263+0000 7f03aaec0640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03a4068cf0 con 0x7f03a409de20 2024-09-13T17:44:06.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.263+0000 7f03aaec0640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a409f090 con 0x7f03a409e220 2024-09-13T17:44:06.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.263+0000 7f039bfff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f03a409de20 0x7f03a4134680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:06.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.263+0000 7f03a8c5c640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f03a4068920 0x7f03a409d710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:37466/0 (socket says 172.21.15.102:37466) 2024-09-13T17:44:06.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.263+0000 7f03a8c5c640 1 -- 172.21.15.102:0/893038945 learned_addr learned my addr 172.21.15.102:0/893038945 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:44:06.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.263+0000 7f03a945d640 1 --2- 172.21.15.102:0/893038945 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f03a409e220 0x7f03a4136bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:06.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.263+0000 7f039bfff640 1 -- 172.21.15.102:0/893038945 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f03a409e220 msgr2=0x7f03a4136bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:06.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.263+0000 7f039bfff640 1 --2- 172.21.15.102:0/893038945 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f03a409e220 0x7f03a4136bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:06.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.263+0000 7f039bfff640 1 -- 172.21.15.102:0/893038945 >> v1:172.21.15.102:6789/0 conn(0x7f03a4068920 legacy=0x7f03a409d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:06.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.263+0000 7f039bfff640 1 -- 172.21.15.102:0/893038945 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a409cce0 con 0x7f03a409de20 2024-09-13T17:44:06.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.263+0000 7f039b7fe640 1 -- 172.21.15.102:0/893038945 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3153512017 0 0) 0x7f03a40695a0 con 0x7f03a4068920 2024-09-13T17:44:06.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.263+0000 7f03a945d640 1 --2- 172.21.15.102:0/893038945 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f03a409e220 0x7f03a4136bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:06.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.264+0000 7f039bfff640 1 --2- 172.21.15.102:0/893038945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03a409de20 0x7f03a4134680 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0394009850 tx=0x7f03940090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d48f0f52a693c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:06.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.264+0000 7f039b7fe640 1 -- 172.21.15.102:0/893038945 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039402ac80 con 0x7f03a409de20 2024-09-13T17:44:06.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.264+0000 7f039b7fe640 1 -- 172.21.15.102:0/893038945 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039402ade0 con 0x7f03a409de20 2024-09-13T17:44:06.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.264+0000 7f039b7fe640 1 -- 172.21.15.102:0/893038945 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0394032670 con 0x7f03a409de20 2024-09-13T17:44:06.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.265+0000 7f03aaec0640 1 -- 172.21.15.102:0/893038945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03a409de20 msgr2=0x7f03a4134680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:06.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.265+0000 7f03aaec0640 1 --2- 172.21.15.102:0/893038945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03a409de20 0x7f03a4134680 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0394009850 tx=0x7f03940090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:06.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.265+0000 7f03aaec0640 1 -- 172.21.15.102:0/893038945 shutdown_connections 2024-09-13T17:44:06.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.265+0000 7f03aaec0640 1 --2- 172.21.15.102:0/893038945 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f03a409e220 0x7f03a4136bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:06.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.265+0000 7f03aaec0640 1 --2- 172.21.15.102:0/893038945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03a409de20 0x7f03a4134680 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:06.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.265+0000 7f03aaec0640 1 -- 172.21.15.102:0/893038945 >> 172.21.15.102:0/893038945 conn(0x7f03a406f320 msgr2=0x7f03a40a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:06.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.265+0000 7f03aaec0640 1 -- 172.21.15.102:0/893038945 shutdown_connections 2024-09-13T17:44:06.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.265+0000 7f03aaec0640 1 -- 172.21.15.102:0/893038945 wait complete. 2024-09-13T17:44:06.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.265+0000 7f03aaec0640 1 Processor -- start 2024-09-13T17:44:06.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.266+0000 7f03aaec0640 1 -- start start 2024-09-13T17:44:06.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.266+0000 7f03aaec0640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f03a4068920 0x7f03a4134100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:06.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.266+0000 7f03aaec0640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03a409de20 0x7f03a4134640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:06.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.266+0000 7f03aaec0640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f03a4134b80 0x7f03a4134fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:06.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.266+0000 7f03aaec0640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a411a210 con 0x7f03a409de20 2024-09-13T17:44:06.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.266+0000 7f03aaec0640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a411ac40 con 0x7f03a4068920 2024-09-13T17:44:06.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.266+0000 7f03aaec0640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f03a4119740 con 0x7f03a4134b80 2024-09-13T17:44:06.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.266+0000 7f03a8c5c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f03a4068920 0x7f03a4134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:06.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.266+0000 7f03a8c5c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f03a4068920 0x7f03a4134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:47010/0 (socket says 172.21.15.102:47010) 2024-09-13T17:44:06.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.266+0000 7f03a8c5c640 1 -- 172.21.15.102:0/29704509 learned_addr learned my addr 172.21.15.102:0/29704509 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:06.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.266+0000 7f039bfff640 1 --2- 172.21.15.102:0/29704509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03a409de20 0x7f03a4134640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:06.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.266+0000 7f03a945d640 1 --2- 172.21.15.102:0/29704509 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f03a4134b80 0x7f03a4134fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:06.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.266+0000 7f03a8c5c640 1 -- 172.21.15.102:0/29704509 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f03a4134b80 msgr2=0x7f03a4134fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:06.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.266+0000 7f03a8c5c640 1 --2- 172.21.15.102:0/29704509 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f03a4134b80 0x7f03a4134fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:06.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.266+0000 7f03a8c5c640 1 -- 172.21.15.102:0/29704509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03a409de20 msgr2=0x7f03a4134640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:06.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.266+0000 7f03a8c5c640 1 --2- 172.21.15.102:0/29704509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03a409de20 0x7f03a4134640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:06.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.266+0000 7f03a8c5c640 1 -- 172.21.15.102:0/29704509 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039c009a20 con 0x7f03a4068920 2024-09-13T17:44:06.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.266+0000 7f039bfff640 1 --2- 172.21.15.102:0/29704509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03a409de20 0x7f03a4134640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:06.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.267+0000 7f03a8c5c640 1 --2- 172.21.15.102:0/29704509 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f03a4068920 0x7f03a4134100 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f03a000c610 tx=0x7f03a0007d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:06.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.267+0000 7f0398ff9640 1 -- 172.21.15.102:0/29704509 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a0005aa0 con 0x7f03a4068920 2024-09-13T17:44:06.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.267+0000 7f0398ff9640 1 -- 172.21.15.102:0/29704509 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a0005c00 con 0x7f03a4068920 2024-09-13T17:44:06.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.267+0000 7f03aaec0640 1 -- 172.21.15.102:0/29704509 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0394009390 con 0x7f03a4068920 2024-09-13T17:44:06.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.267+0000 7f0398ff9640 1 -- 172.21.15.102:0/29704509 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a00070d0 con 0x7f03a4068920 2024-09-13T17:44:06.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.267+0000 7f03aaec0640 1 -- 172.21.15.102:0/29704509 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a4135510 con 0x7f03a4068920 2024-09-13T17:44:06.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.268+0000 7f0398ff9640 1 -- 172.21.15.102:0/29704509 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f03a0007500 con 0x7f03a4068920 2024-09-13T17:44:06.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.268+0000 7f0398ff9640 1 -- 172.21.15.102:0/29704509 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 2808+0+0 (secure 0 0 0) 0x7f03a0041110 con 0x7f03a4068920 2024-09-13T17:44:06.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.268+0000 7f03aaec0640 1 -- 172.21.15.102:0/29704509 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03a4119210 con 0x7f03a4068920 2024-09-13T17:44:06.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.272+0000 7f0398ff9640 1 -- 172.21.15.102:0/29704509 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f03a0017070 con 0x7f03a4068920 2024-09-13T17:44:06.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.427+0000 7f03aaec0640 1 -- 172.21.15.102:0/29704509 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f03a411a3c0 con 0x7f03a4068920 2024-09-13T17:44:06.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.427+0000 7f0398ff9640 1 -- 172.21.15.102:0/29704509 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v28) v1 ==== 84+0+126 (secure 0 0 0) 0x7f03a001a360 con 0x7f03a4068920 2024-09-13T17:44:06.429 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:44:06.429 INFO:teuthology.orchestra.run.smithi102.stdout:[ 2024-09-13T17:44:06.429 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-13T17:44:06.429 INFO:teuthology.orchestra.run.smithi102.stdout: "poolnum": 1, 2024-09-13T17:44:06.430 INFO:teuthology.orchestra.run.smithi102.stdout: "poolname": ".mgr" 2024-09-13T17:44:06.430 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-13T17:44:06.430 INFO:teuthology.orchestra.run.smithi102.stdout: { 2024-09-13T17:44:06.430 INFO:teuthology.orchestra.run.smithi102.stdout: "poolnum": 2, 2024-09-13T17:44:06.430 INFO:teuthology.orchestra.run.smithi102.stdout: "poolname": "rbd" 2024-09-13T17:44:06.430 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-13T17:44:06.430 INFO:teuthology.orchestra.run.smithi102.stdout:] 2024-09-13T17:44:06.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.429+0000 7f03aaec0640 1 -- 172.21.15.102:0/29704509 >> v1:172.21.15.102:6816/34937 conn(0x7f0390031190 legacy=0x7f0390033640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:06.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.429+0000 7f03aaec0640 1 -- 172.21.15.102:0/29704509 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f03a4068920 msgr2=0x7f03a4134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:06.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.429+0000 7f03aaec0640 1 --2- 172.21.15.102:0/29704509 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f03a4068920 0x7f03a4134100 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f03a000c610 tx=0x7f03a0007d00 comp rx=0 tx=0).stop 2024-09-13T17:44:06.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.429+0000 7f03aaec0640 1 -- 172.21.15.102:0/29704509 shutdown_connections 2024-09-13T17:44:06.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.429+0000 7f03aaec0640 1 --2- 172.21.15.102:0/29704509 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f03a4134b80 0x7f03a4134fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:06.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.429+0000 7f03aaec0640 1 --2- 172.21.15.102:0/29704509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03a409de20 0x7f03a4134640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:06.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.429+0000 7f03aaec0640 1 --2- 172.21.15.102:0/29704509 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f03a4068920 0x7f03a4134100 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:06.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.429+0000 7f03aaec0640 1 -- 172.21.15.102:0/29704509 >> 172.21.15.102:0/29704509 conn(0x7f03a406f320 msgr2=0x7f03a409bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:06.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.429+0000 7f03aaec0640 1 -- 172.21.15.102:0/29704509 shutdown_connections 2024-09-13T17:44:06.433 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.429+0000 7f03aaec0640 1 -- 172.21.15.102:0/29704509 wait complete. 2024-09-13T17:44:06.441 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-13T17:44:06.441 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-13T17:44:06.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.536+0000 7f74eb64f640 1 Processor -- start 2024-09-13T17:44:06.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.536+0000 7f74eb64f640 1 -- start start 2024-09-13T17:44:06.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.537+0000 7f74eb64f640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74e409a620 0x7f74e4096e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:06.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.537+0000 7f74eb64f640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f74e40977a0 0x7f74e412d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:06.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.537+0000 7f74eb64f640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74e4097be0 con 0x7f74e40973d0 2024-09-13T17:44:06.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.537+0000 7f74eb64f640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74e406a7a0 con 0x7f74e40977a0 2024-09-13T17:44:06.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.537+0000 7f74eb64f640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e40981e0 con 0x7f74e409a620 2024-09-13T17:44:06.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.537+0000 7f74e93eb640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74e409a620 0x7f74e4096e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:06.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.537+0000 7f74e9bec640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f74e40977a0 0x7f74e412d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:06.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.537+0000 7f74e93eb640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74e409a620 0x7f74e4096e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:47012/0 (socket says 172.21.15.102:47012) 2024-09-13T17:44:06.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.537+0000 7f74e8bea640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f74e40973d0 0x7f74e412d280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:37470/0 (socket says 172.21.15.102:37470) 2024-09-13T17:44:06.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.537+0000 7f74e9bec640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f74e40977a0 0x7f74e412d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34422/0 (socket says 172.21.15.102:34422) 2024-09-13T17:44:06.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.537+0000 7f74e93eb640 1 -- 172.21.15.102:0/1402143131 learned_addr learned my addr 172.21.15.102:0/1402143131 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:06.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.537+0000 7f74e9bec640 1 -- 172.21.15.102:0/1402143131 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74e409a620 msgr2=0x7f74e4096e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:06.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.537+0000 7f74e9bec640 1 --2- 172.21.15.102:0/1402143131 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74e409a620 0x7f74e4096e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:06.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.537+0000 7f74e9bec640 1 -- 172.21.15.102:0/1402143131 >> v1:172.21.15.102:6789/0 conn(0x7f74e40973d0 legacy=0x7f74e412d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:06.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.537+0000 7f74dbfff640 1 -- 172.21.15.102:0/1402143131 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2644700598 0 0) 0x7f74d4001110 con 0x7f74e40973d0 2024-09-13T17:44:06.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.537+0000 7f74e9bec640 1 -- 172.21.15.102:0/1402143131 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e4096b80 con 0x7f74e40977a0 2024-09-13T17:44:06.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.537+0000 7f74e93eb640 1 --2- 172.21.15.102:0/1402143131 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74e409a620 0x7f74e4096e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:06.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.538+0000 7f74e9bec640 1 --2- 172.21.15.102:0/1402143131 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74e40977a0 0x7f74e412d990 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f74dc009e60 tx=0x7f74dc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5da08a38375e7ab server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:06.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.538+0000 7f74dbfff640 1 -- 172.21.15.102:0/1402143131 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74dc007550 con 0x7f74e40977a0 2024-09-13T17:44:06.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.538+0000 7f74dbfff640 1 -- 172.21.15.102:0/1402143131 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74dc02d050 con 0x7f74e40977a0 2024-09-13T17:44:06.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.538+0000 7f74dbfff640 1 -- 172.21.15.102:0/1402143131 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74dc0078e0 con 0x7f74e40977a0 2024-09-13T17:44:06.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.538+0000 7f74eb64f640 1 -- 172.21.15.102:0/1402143131 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74e40977a0 msgr2=0x7f74e412d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:06.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.538+0000 7f74eb64f640 1 --2- 172.21.15.102:0/1402143131 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74e40977a0 0x7f74e412d990 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f74dc009e60 tx=0x7f74dc009780 comp rx=0 tx=0).stop 2024-09-13T17:44:06.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.539+0000 7f74eb64f640 1 -- 172.21.15.102:0/1402143131 shutdown_connections 2024-09-13T17:44:06.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.539+0000 7f74eb64f640 1 --2- 172.21.15.102:0/1402143131 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74e40977a0 0x7f74e412d990 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:06.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.539+0000 7f74eb64f640 1 --2- 172.21.15.102:0/1402143131 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74e409a620 0x7f74e4096e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:06.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.539+0000 7f74eb64f640 1 -- 172.21.15.102:0/1402143131 >> 172.21.15.102:0/1402143131 conn(0x7f74e409e720 msgr2=0x7f74e409eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:06.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.539+0000 7f74eb64f640 1 -- 172.21.15.102:0/1402143131 shutdown_connections 2024-09-13T17:44:06.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.539+0000 7f74eb64f640 1 -- 172.21.15.102:0/1402143131 wait complete. 2024-09-13T17:44:06.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.539+0000 7f74eb64f640 1 Processor -- start 2024-09-13T17:44:06.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.539+0000 7f74eb64f640 1 -- start start 2024-09-13T17:44:06.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.540+0000 7f74eb64f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74e40973d0 0x7f74e413a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:06.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.540+0000 7f74eb64f640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74e40977a0 0x7f74e413ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:06.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.540+0000 7f74eb64f640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f74e409a620 0x7f74e41161e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:06.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.540+0000 7f74eb64f640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e4118750 con 0x7f74e40973d0 2024-09-13T17:44:06.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.540+0000 7f74eb64f640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e4069dd0 con 0x7f74e40977a0 2024-09-13T17:44:06.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.540+0000 7f74eb64f640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74e4119620 con 0x7f74e409a620 2024-09-13T17:44:06.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.540+0000 7f74e93eb640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74e40973d0 0x7f74e413a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:06.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.540+0000 7f74e93eb640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74e40973d0 0x7f74e413a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34438/0 (socket says 172.21.15.102:34438) 2024-09-13T17:44:06.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.540+0000 7f74e93eb640 1 -- 172.21.15.102:0/3500288028 learned_addr learned my addr 172.21.15.102:0/3500288028 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:06.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.540+0000 7f74e8bea640 1 --2- 172.21.15.102:0/3500288028 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74e40977a0 0x7f74e413ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:06.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.540+0000 7f74e9bec640 1 --2- 172.21.15.102:0/3500288028 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f74e409a620 0x7f74e41161e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:06.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.540+0000 7f74e93eb640 1 -- 172.21.15.102:0/3500288028 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f74e409a620 msgr2=0x7f74e41161e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:06.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.540+0000 7f74e93eb640 1 --2- 172.21.15.102:0/3500288028 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f74e409a620 0x7f74e41161e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:06.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.540+0000 7f74e93eb640 1 -- 172.21.15.102:0/3500288028 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74e40977a0 msgr2=0x7f74e413ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:06.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.540+0000 7f74e93eb640 1 --2- 172.21.15.102:0/3500288028 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74e40977a0 0x7f74e413ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:06.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.540+0000 7f74e93eb640 1 -- 172.21.15.102:0/3500288028 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e000d900 con 0x7f74e40973d0 2024-09-13T17:44:06.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.540+0000 7f74e8bea640 1 --2- 172.21.15.102:0/3500288028 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74e40977a0 0x7f74e413ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:06.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.541+0000 7f74e93eb640 1 --2- 172.21.15.102:0/3500288028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74e40973d0 0x7f74e413a910 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f74e000df10 tx=0x7f74e000f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:06.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.541+0000 7f74d97fa640 1 -- 172.21.15.102:0/3500288028 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e000dc70 con 0x7f74e40973d0 2024-09-13T17:44:06.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.541+0000 7f74d97fa640 1 -- 172.21.15.102:0/3500288028 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e0013040 con 0x7f74e40973d0 2024-09-13T17:44:06.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.541+0000 7f74eb64f640 1 -- 172.21.15.102:0/3500288028 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74dc009a20 con 0x7f74e40973d0 2024-09-13T17:44:06.546 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.541+0000 7f74d97fa640 1 -- 172.21.15.102:0/3500288028 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e0010dd0 con 0x7f74e40973d0 2024-09-13T17:44:06.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.541+0000 7f74eb64f640 1 -- 172.21.15.102:0/3500288028 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e411a310 con 0x7f74e40973d0 2024-09-13T17:44:06.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.541+0000 7f74d97fa640 1 -- 172.21.15.102:0/3500288028 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f74e0018690 con 0x7f74e40973d0 2024-09-13T17:44:06.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.542+0000 7f74d97fa640 1 -- 172.21.15.102:0/3500288028 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 2808+0+0 (secure 0 0 0) 0x7f74e0045df0 con 0x7f74e40973d0 2024-09-13T17:44:06.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.542+0000 7f74eb64f640 1 -- 172.21.15.102:0/3500288028 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74e4098900 con 0x7f74e40973d0 2024-09-13T17:44:06.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.546+0000 7f74d97fa640 1 -- 172.21.15.102:0/3500288028 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f74e000b450 con 0x7f74e40973d0 2024-09-13T17:44:06.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:06.700+0000 7f74eb64f640 1 -- 172.21.15.102:0/3500288028 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f74e4064000 con 0x7f74e40973d0 2024-09-13T17:44:07.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.053+0000 7f74d97fa640 1 -- 172.21.15.102:0/3500288028 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7f74e0016e60 con 0x7f74e40973d0 2024-09-13T17:44:07.055 INFO:teuthology.orchestra.run.smithi102.stderr:pool 'cephfs_metadata' created 2024-09-13T17:44:07.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.058+0000 7f74eb64f640 1 -- 172.21.15.102:0/3500288028 >> v1:172.21.15.102:6816/34937 conn(0x7f74c8031160 legacy=0x7f74c8033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:07.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.058+0000 7f74eb64f640 1 -- 172.21.15.102:0/3500288028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74e40973d0 msgr2=0x7f74e413a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:07.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.058+0000 7f74eb64f640 1 --2- 172.21.15.102:0/3500288028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74e40973d0 0x7f74e413a910 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f74e000df10 tx=0x7f74e000f040 comp rx=0 tx=0).stop 2024-09-13T17:44:07.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.058+0000 7f74eb64f640 1 -- 172.21.15.102:0/3500288028 shutdown_connections 2024-09-13T17:44:07.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.058+0000 7f74eb64f640 1 --2- 172.21.15.102:0/3500288028 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f74e409a620 0x7f74e41161e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:07.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.058+0000 7f74eb64f640 1 --2- 172.21.15.102:0/3500288028 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f74e40977a0 0x7f74e413ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:07.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.058+0000 7f74eb64f640 1 --2- 172.21.15.102:0/3500288028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74e40973d0 0x7f74e413a910 secure :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f74e000df10 tx=0x7f74e000f040 comp rx=0 tx=0).stop 2024-09-13T17:44:07.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.058+0000 7f74eb64f640 1 -- 172.21.15.102:0/3500288028 >> 172.21.15.102:0/3500288028 conn(0x7f74e409e720 msgr2=0x7f74e412e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:07.061 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.060+0000 7f74eb64f640 1 -- 172.21.15.102:0/3500288028 shutdown_connections 2024-09-13T17:44:07.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.061+0000 7f74eb64f640 1 -- 172.21.15.102:0/3500288028 wait complete. 2024-09-13T17:44:07.074 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-13T17:44:07.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.173+0000 7fe844b9d640 1 Processor -- start 2024-09-13T17:44:07.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.173+0000 7fe844b9d640 1 -- start start 2024-09-13T17:44:07.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.174+0000 7fe844b9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe84009c4b0 0x7fe84009a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:07.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.174+0000 7fe844b9d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe84009ae50 0x7fe84009b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:07.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.174+0000 7fe844b9d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe84005df70 con 0x7fe8401340c0 2024-09-13T17:44:07.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.174+0000 7fe844b9d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe840096d30 con 0x7fe84009ae50 2024-09-13T17:44:07.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.174+0000 7fe844b9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe840097320 con 0x7fe84009c4b0 2024-09-13T17:44:07.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.174+0000 7fe83dd9b640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe84009ae50 0x7fe84009b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:07.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.174+0000 7fe83e59c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe84009c4b0 0x7fe84009a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:07.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.174+0000 7fe83dd9b640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe84009ae50 0x7fe84009b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34448/0 (socket says 172.21.15.102:34448) 2024-09-13T17:44:07.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.174+0000 7fe83dd9b640 1 -- 172.21.15.102:0/717705362 learned_addr learned my addr 172.21.15.102:0/717705362 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:07.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.174+0000 7fe83e59c640 1 --2- 172.21.15.102:0/717705362 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe84009c4b0 0x7fe84009a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:47020/0 (socket says 172.21.15.102:47020) 2024-09-13T17:44:07.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.174+0000 7fe83e59c640 1 -- 172.21.15.102:0/717705362 >> v2:172.21.15.102:3300/0 conn(0x7fe84009ae50 msgr2=0x7fe84009b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:07.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.174+0000 7fe83e59c640 1 --2- 172.21.15.102:0/717705362 >> v2:172.21.15.102:3300/0 conn(0x7fe84009ae50 0x7fe84009b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:07.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.174+0000 7fe83e59c640 1 -- 172.21.15.102:0/717705362 >> v1:172.21.15.102:6789/0 conn(0x7fe8401340c0 legacy=0x7fe8401364a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:07.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.174+0000 7fe83d59a640 1 -- 172.21.15.102:0/717705362 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1681670146 0 0) 0x7fe84005df70 con 0x7fe8401340c0 2024-09-13T17:44:07.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.174+0000 7fe83e59c640 1 -- 172.21.15.102:0/717705362 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe84009b760 con 0x7fe84009c4b0 2024-09-13T17:44:07.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.174+0000 7fe83dd9b640 1 --2- 172.21.15.102:0/717705362 >> v2:172.21.15.102:3300/0 conn(0x7fe84009ae50 0x7fe84009b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:07.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.175+0000 7fe83e59c640 1 --2- 172.21.15.102:0/717705362 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe84009c4b0 0x7fe84009a910 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe830009880 tx=0x7fe8300090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6549893568797875 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:07.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.175+0000 7fe83d59a640 1 -- 172.21.15.102:0/717705362 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83002ad70 con 0x7fe84009c4b0 2024-09-13T17:44:07.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.175+0000 7fe83d59a640 1 -- 172.21.15.102:0/717705362 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe830032440 con 0x7fe84009c4b0 2024-09-13T17:44:07.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.175+0000 7fe83d59a640 1 -- 172.21.15.102:0/717705362 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8300327e0 con 0x7fe84009c4b0 2024-09-13T17:44:07.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.176+0000 7fe844b9d640 1 -- 172.21.15.102:0/717705362 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe84009c4b0 msgr2=0x7fe84009a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:07.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.176+0000 7fe844b9d640 1 --2- 172.21.15.102:0/717705362 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe84009c4b0 0x7fe84009a910 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe830009880 tx=0x7fe8300090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:07.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.176+0000 7fe844b9d640 1 -- 172.21.15.102:0/717705362 shutdown_connections 2024-09-13T17:44:07.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.176+0000 7fe844b9d640 1 --2- 172.21.15.102:0/717705362 >> v2:172.21.15.102:3300/0 conn(0x7fe84009ae50 0x7fe84009b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:07.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.176+0000 7fe844b9d640 1 --2- 172.21.15.102:0/717705362 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe84009c4b0 0x7fe84009a910 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:07.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.176+0000 7fe844b9d640 1 -- 172.21.15.102:0/717705362 >> 172.21.15.102:0/717705362 conn(0x7fe84009f960 msgr2=0x7fe84009ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:07.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.176+0000 7fe844b9d640 1 -- 172.21.15.102:0/717705362 shutdown_connections 2024-09-13T17:44:07.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.176+0000 7fe844b9d640 1 -- 172.21.15.102:0/717705362 wait complete. 2024-09-13T17:44:07.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.176+0000 7fe844b9d640 1 Processor -- start 2024-09-13T17:44:07.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.176+0000 7fe844b9d640 1 -- start start 2024-09-13T17:44:07.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.177+0000 7fe844b9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe84009ae50 0x7fe840094df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:07.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.177+0000 7fe844b9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe84009c4b0 0x7fe84008c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:07.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.177+0000 7fe844b9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8401340c0 0x7fe84008c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:07.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.177+0000 7fe844b9d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84008df70 con 0x7fe8401340c0 2024-09-13T17:44:07.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.177+0000 7fe844b9d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe84008e9b0 con 0x7fe84009ae50 2024-09-13T17:44:07.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.177+0000 7fe844b9d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe840095330 con 0x7fe84009c4b0 2024-09-13T17:44:07.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.177+0000 7fe83e59c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe84009ae50 0x7fe840094df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:07.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.177+0000 7fe83dd9b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe84009c4b0 0x7fe84008c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:07.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.177+0000 7fe83ed9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8401340c0 0x7fe84008c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:07.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.177+0000 7fe83e59c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe84009ae50 0x7fe840094df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:47024/0 (socket says 172.21.15.102:47024) 2024-09-13T17:44:07.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.177+0000 7fe83ed9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8401340c0 0x7fe84008c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34458/0 (socket says 172.21.15.102:34458) 2024-09-13T17:44:07.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.177+0000 7fe83dd9b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe84009c4b0 0x7fe84008c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54302/0 (socket says 172.21.15.102:54302) 2024-09-13T17:44:07.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.177+0000 7fe83e59c640 1 -- 172.21.15.102:0/1158583837 learned_addr learned my addr 172.21.15.102:0/1158583837 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:07.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.177+0000 7fe83e59c640 1 -- 172.21.15.102:0/1158583837 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe84009c4b0 msgr2=0x7fe84008c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:07.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.177+0000 7fe83e59c640 1 --2- 172.21.15.102:0/1158583837 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe84009c4b0 0x7fe84008c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:07.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.177+0000 7fe83e59c640 1 -- 172.21.15.102:0/1158583837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8401340c0 msgr2=0x7fe84008c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:07.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.177+0000 7fe83e59c640 1 --2- 172.21.15.102:0/1158583837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8401340c0 0x7fe84008c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:07.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.177+0000 7fe83e59c640 1 -- 172.21.15.102:0/1158583837 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe834009450 con 0x7fe84009ae50 2024-09-13T17:44:07.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.177+0000 7fe83e59c640 1 --2- 172.21.15.102:0/1158583837 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe84009ae50 0x7fe840094df0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe830009360 tx=0x7fe830032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:07.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.178+0000 7fe82e7fc640 1 -- 172.21.15.102:0/1158583837 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe830000d10 con 0x7fe84009ae50 2024-09-13T17:44:07.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.178+0000 7fe82e7fc640 1 -- 172.21.15.102:0/1158583837 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe830031a60 con 0x7fe84009ae50 2024-09-13T17:44:07.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.178+0000 7fe82e7fc640 1 -- 172.21.15.102:0/1158583837 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe830031e80 con 0x7fe84009ae50 2024-09-13T17:44:07.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.178+0000 7fe844b9d640 1 -- 172.21.15.102:0/1158583837 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe830009390 con 0x7fe84009ae50 2024-09-13T17:44:07.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.178+0000 7fe844b9d640 1 -- 172.21.15.102:0/1158583837 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe84008f660 con 0x7fe84009ae50 2024-09-13T17:44:07.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.178+0000 7fe82e7fc640 1 -- 172.21.15.102:0/1158583837 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fe83003c610 con 0x7fe84009ae50 2024-09-13T17:44:07.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.179+0000 7fe82e7fc640 1 -- 172.21.15.102:0/1158583837 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 3178+0+0 (secure 0 0 0) 0x7fe830062850 con 0x7fe84009ae50 2024-09-13T17:44:07.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.179+0000 7fe844b9d640 1 -- 172.21.15.102:0/1158583837 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe840064000 con 0x7fe84009ae50 2024-09-13T17:44:07.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.183+0000 7fe82e7fc640 1 -- 172.21.15.102:0/1158583837 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fe830037080 con 0x7fe84009ae50 2024-09-13T17:44:07.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:07.336+0000 7fe844b9d640 1 -- 172.21.15.102:0/1158583837 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe840098ea0 con 0x7fe84009ae50 2024-09-13T17:44:08.059 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.057+0000 7fe82e7fc640 1 -- 172.21.15.102:0/1158583837 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v30) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe830049020 con 0x7fe84009ae50 2024-09-13T17:44:08.059 INFO:teuthology.orchestra.run.smithi102.stderr:pool 'cephfs_data' created 2024-09-13T17:44:08.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.060+0000 7fe844b9d640 1 -- 172.21.15.102:0/1158583837 >> v1:172.21.15.102:6816/34937 conn(0x7fe820031160 legacy=0x7fe820033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:08.062 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.060+0000 7fe844b9d640 1 -- 172.21.15.102:0/1158583837 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe84009ae50 msgr2=0x7fe840094df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:08.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.060+0000 7fe844b9d640 1 --2- 172.21.15.102:0/1158583837 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe84009ae50 0x7fe840094df0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe830009360 tx=0x7fe830032000 comp rx=0 tx=0).stop 2024-09-13T17:44:08.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.060+0000 7fe844b9d640 1 -- 172.21.15.102:0/1158583837 shutdown_connections 2024-09-13T17:44:08.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.060+0000 7fe844b9d640 1 --2- 172.21.15.102:0/1158583837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8401340c0 0x7fe84008c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:08.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.060+0000 7fe844b9d640 1 --2- 172.21.15.102:0/1158583837 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe84009c4b0 0x7fe84008c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:08.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.060+0000 7fe844b9d640 1 --2- 172.21.15.102:0/1158583837 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe84009ae50 0x7fe840094df0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:08.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.060+0000 7fe844b9d640 1 -- 172.21.15.102:0/1158583837 >> 172.21.15.102:0/1158583837 conn(0x7fe84009f960 msgr2=0x7fe840133610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:08.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.063+0000 7fe844b9d640 1 -- 172.21.15.102:0/1158583837 shutdown_connections 2024-09-13T17:44:08.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.064+0000 7fe844b9d640 1 -- 172.21.15.102:0/1158583837 wait complete. 2024-09-13T17:44:08.096 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-13T17:44:08.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.192+0000 7f24e5c94640 1 Processor -- start 2024-09-13T17:44:08.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.192+0000 7f24e5c94640 1 -- start start 2024-09-13T17:44:08.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.193+0000 7f24e5c94640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f24e00689d0 0x7f24e009d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:08.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.193+0000 7f24e5c94640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f24e009dd70 0x7f24e012f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:08.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.193+0000 7f24e5c94640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f24e009e160 0x7f24e012fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:08.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.193+0000 7f24e5c94640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24e0096db0 con 0x7f24e009e160 2024-09-13T17:44:08.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.193+0000 7f24e5c94640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e0097730 con 0x7f24e009dd70 2024-09-13T17:44:08.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.193+0000 7f24e5c94640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24e0096b10 con 0x7f24e00689d0 2024-09-13T17:44:08.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.193+0000 7f24deffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f24e009dd70 0x7f24e012f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:08.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.193+0000 7f24df7fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f24e00689d0 0x7f24e009d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:08.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.193+0000 7f24dffff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f24e009e160 0x7f24e012fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:08.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.193+0000 7f24deffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f24e009dd70 0x7f24e012f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:47034/0 (socket says 172.21.15.102:47034) 2024-09-13T17:44:08.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.193+0000 7f24df7fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f24e00689d0 0x7f24e009d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54304/0 (socket says 172.21.15.102:54304) 2024-09-13T17:44:08.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.193+0000 7f24deffd640 1 -- 172.21.15.102:0/1142459686 learned_addr learned my addr 172.21.15.102:0/1142459686 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:08.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.193+0000 7f24deffd640 1 -- 172.21.15.102:0/1142459686 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f24e00689d0 msgr2=0x7f24e009d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:08.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.193+0000 7f24deffd640 1 --2- 172.21.15.102:0/1142459686 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f24e00689d0 0x7f24e009d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:08.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.193+0000 7f24deffd640 1 -- 172.21.15.102:0/1142459686 >> v2:172.21.15.102:3300/0 conn(0x7f24e009e160 msgr2=0x7f24e012fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:08.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.193+0000 7f24deffd640 1 --2- 172.21.15.102:0/1142459686 >> v2:172.21.15.102:3300/0 conn(0x7f24e009e160 0x7f24e012fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:08.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.193+0000 7f24deffd640 1 -- 172.21.15.102:0/1142459686 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e00696e0 con 0x7f24e009dd70 2024-09-13T17:44:08.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.194+0000 7f24deffd640 1 --2- 172.21.15.102:0/1142459686 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f24e009dd70 0x7f24e012f610 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f24d4009940 tx=0x7f24d40091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d770a6dcec1aa73 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:08.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.194+0000 7f24de7fc640 1 -- 172.21.15.102:0/1142459686 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d402ac70 con 0x7f24e009dd70 2024-09-13T17:44:08.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.194+0000 7f24de7fc640 1 -- 172.21.15.102:0/1142459686 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d402add0 con 0x7f24e009dd70 2024-09-13T17:44:08.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.194+0000 7f24de7fc640 1 -- 172.21.15.102:0/1142459686 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d4032670 con 0x7f24e009dd70 2024-09-13T17:44:08.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.195+0000 7f24e5c94640 1 -- 172.21.15.102:0/1142459686 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f24e009dd70 msgr2=0x7f24e012f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:08.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.195+0000 7f24e5c94640 1 --2- 172.21.15.102:0/1142459686 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f24e009dd70 0x7f24e012f610 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f24d4009940 tx=0x7f24d40091d0 comp rx=0 tx=0).stop 2024-09-13T17:44:08.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.195+0000 7f24e5c94640 1 -- 172.21.15.102:0/1142459686 shutdown_connections 2024-09-13T17:44:08.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.195+0000 7f24e5c94640 1 --2- 172.21.15.102:0/1142459686 >> v2:172.21.15.102:3300/0 conn(0x7f24e009e160 0x7f24e012fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:08.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.195+0000 7f24e5c94640 1 --2- 172.21.15.102:0/1142459686 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f24e009dd70 0x7f24e012f610 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:08.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.195+0000 7f24e5c94640 1 --2- 172.21.15.102:0/1142459686 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f24e00689d0 0x7f24e009d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:08.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.195+0000 7f24e5c94640 1 -- 172.21.15.102:0/1142459686 >> 172.21.15.102:0/1142459686 conn(0x7f24e006f320 msgr2=0x7f24e00a2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:08.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.195+0000 7f24e5c94640 1 -- 172.21.15.102:0/1142459686 shutdown_connections 2024-09-13T17:44:08.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.195+0000 7f24e5c94640 1 -- 172.21.15.102:0/1142459686 wait complete. 2024-09-13T17:44:08.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.196+0000 7f24e5c94640 1 Processor -- start 2024-09-13T17:44:08.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.196+0000 7f24e5c94640 1 -- start start 2024-09-13T17:44:08.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.196+0000 7f24e5c94640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f24e00689d0 0x7f24e012f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:08.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.196+0000 7f24e5c94640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f24e009dd70 0x7f24e011e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:08.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.196+0000 7f24e5c94640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24e009e160 0x7f24e011ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:08.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.196+0000 7f24e5c94640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e0120800 con 0x7f24e009e160 2024-09-13T17:44:08.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.196+0000 7f24e5c94640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e0120440 con 0x7f24e009dd70 2024-09-13T17:44:08.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.196+0000 7f24e5c94640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24e01211d0 con 0x7f24e00689d0 2024-09-13T17:44:08.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.196+0000 7f24df7fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f24e00689d0 0x7f24e012f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:08.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.196+0000 7f24deffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f24e009dd70 0x7f24e011e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:08.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.196+0000 7f24dffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24e009e160 0x7f24e011ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:08.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.196+0000 7f24df7fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f24e00689d0 0x7f24e012f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54310/0 (socket says 172.21.15.102:54310) 2024-09-13T17:44:08.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.196+0000 7f24df7fe640 1 -- 172.21.15.102:0/3390475692 learned_addr learned my addr 172.21.15.102:0/3390475692 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:08.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.197+0000 7f24deffd640 1 -- 172.21.15.102:0/3390475692 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f24e00689d0 msgr2=0x7f24e012f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:08.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.197+0000 7f24deffd640 1 --2- 172.21.15.102:0/3390475692 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f24e00689d0 0x7f24e012f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:08.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.197+0000 7f24deffd640 1 -- 172.21.15.102:0/3390475692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24e009e160 msgr2=0x7f24e011ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:08.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.197+0000 7f24deffd640 1 --2- 172.21.15.102:0/3390475692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24e009e160 0x7f24e011ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:08.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.197+0000 7f24deffd640 1 -- 172.21.15.102:0/3390475692 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d4009450 con 0x7f24e009dd70 2024-09-13T17:44:08.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.197+0000 7f24df7fe640 1 --2- 172.21.15.102:0/3390475692 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f24e00689d0 0x7f24e012f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:08.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.197+0000 7f24dffff640 1 --2- 172.21.15.102:0/3390475692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24e009e160 0x7f24e011ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:08.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.197+0000 7f24deffd640 1 --2- 172.21.15.102:0/3390475692 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f24e009dd70 0x7f24e011e790 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f24d4031000 tx=0x7f24d4032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:08.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.197+0000 7f24c77fe640 1 -- 172.21.15.102:0/3390475692 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d403d650 con 0x7f24e009dd70 2024-09-13T17:44:08.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.197+0000 7f24c77fe640 1 -- 172.21.15.102:0/3390475692 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d403d7d0 con 0x7f24e009dd70 2024-09-13T17:44:08.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.197+0000 7f24c77fe640 1 -- 172.21.15.102:0/3390475692 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d403db90 con 0x7f24e009dd70 2024-09-13T17:44:08.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.197+0000 7f24e5c94640 1 -- 172.21.15.102:0/3390475692 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24e0121ef0 con 0x7f24e009dd70 2024-09-13T17:44:08.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.197+0000 7f24e5c94640 1 -- 172.21.15.102:0/3390475692 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e0122890 con 0x7f24e009dd70 2024-09-13T17:44:08.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.198+0000 7f24c77fe640 1 -- 172.21.15.102:0/3390475692 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f24d4039040 con 0x7f24e009dd70 2024-09-13T17:44:08.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.198+0000 7f24c77fe640 1 -- 172.21.15.102:0/3390475692 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3560+0+0 (secure 0 0 0) 0x7f24d40617c0 con 0x7f24e009dd70 2024-09-13T17:44:08.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.198+0000 7f24e5c94640 1 -- 172.21.15.102:0/3390475692 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24e009ebc0 con 0x7f24e009dd70 2024-09-13T17:44:08.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.202+0000 7f24c77fe640 1 -- 172.21.15.102:0/3390475692 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f24d4037020 con 0x7f24e009dd70 2024-09-13T17:44:08.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:08.383+0000 7f24e5c94640 1 -- 172.21.15.102:0/3390475692 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f24e0122ae0 con 0x7f24e009dd70 2024-09-13T17:44:09.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.073+0000 7f24c77fe640 1 -- 172.21.15.102:0/3390475692 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v9) v1 ==== 172+0+0 (secure 0 0 0) 0x7f24e0122ae0 con 0x7f24e009dd70 2024-09-13T17:44:09.076 INFO:teuthology.orchestra.run.smithi102.stderr:new fs with metadata pool 5 and data pool 6 2024-09-13T17:44:09.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.079+0000 7f24e5c94640 1 -- 172.21.15.102:0/3390475692 >> v1:172.21.15.102:6816/34937 conn(0x7f24cc031160 legacy=0x7f24cc033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:09.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.079+0000 7f24e5c94640 1 -- 172.21.15.102:0/3390475692 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f24e009dd70 msgr2=0x7f24e011e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:09.082 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.079+0000 7f24e5c94640 1 --2- 172.21.15.102:0/3390475692 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f24e009dd70 0x7f24e011e790 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f24d4031000 tx=0x7f24d4032000 comp rx=0 tx=0).stop 2024-09-13T17:44:09.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.080+0000 7f24e5c94640 1 -- 172.21.15.102:0/3390475692 shutdown_connections 2024-09-13T17:44:09.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.080+0000 7f24e5c94640 1 --2- 172.21.15.102:0/3390475692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24e009e160 0x7f24e011ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:09.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.080+0000 7f24e5c94640 1 --2- 172.21.15.102:0/3390475692 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f24e009dd70 0x7f24e011e790 secure :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f24d4031000 tx=0x7f24d4032000 comp rx=0 tx=0).stop 2024-09-13T17:44:09.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.080+0000 7f24e5c94640 1 --2- 172.21.15.102:0/3390475692 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f24e00689d0 0x7f24e012f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:09.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.080+0000 7f24e5c94640 1 -- 172.21.15.102:0/3390475692 >> 172.21.15.102:0/3390475692 conn(0x7f24e006f320 msgr2=0x7f24e009b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:09.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.082+0000 7f24e5c94640 1 -- 172.21.15.102:0/3390475692 shutdown_connections 2024-09-13T17:44:09.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.082+0000 7f24e5c94640 1 -- 172.21.15.102:0/3390475692 wait complete. 2024-09-13T17:44:09.105 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:44:09.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.197+0000 7f0a8355d640 1 Processor -- start 2024-09-13T17:44:09.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.197+0000 7f0a8355d640 1 -- start start 2024-09-13T17:44:09.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.198+0000 7f0a8355d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0a7c096a70 0x7f0a7c12c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:09.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.198+0000 7f0a8355d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0a7c12c8b0 0x7f0a7c098410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:09.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.198+0000 7f0a8355d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0a7c12cd00 0x7f0a7c098950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:09.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.198+0000 7f0a8355d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a7c06a1c0 con 0x7f0a7c12cd00 2024-09-13T17:44:09.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.198+0000 7f0a8355d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c09a970 con 0x7f0a7c12c8b0 2024-09-13T17:44:09.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.198+0000 7f0a8355d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c0999f0 con 0x7f0a7c096a70 2024-09-13T17:44:09.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.198+0000 7f0a80af8640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0a7c12c8b0 0x7f0a7c098410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:09.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.198+0000 7f0a81afa640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0a7c12cd00 0x7f0a7c098950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:09.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.198+0000 7f0a812f9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0a7c096a70 0x7f0a7c12c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:09.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.198+0000 7f0a80af8640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0a7c12c8b0 0x7f0a7c098410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:47046/0 (socket says 172.21.15.102:47046) 2024-09-13T17:44:09.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.198+0000 7f0a80af8640 1 -- 172.21.15.102:0/2404769545 learned_addr learned my addr 172.21.15.102:0/2404769545 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:09.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.198+0000 7f0a812f9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0a7c096a70 0x7f0a7c12c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54326/0 (socket says 172.21.15.102:54326) 2024-09-13T17:44:09.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.198+0000 7f0a80af8640 1 -- 172.21.15.102:0/2404769545 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0a7c096a70 msgr2=0x7f0a7c12c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:09.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.198+0000 7f0a80af8640 1 --2- 172.21.15.102:0/2404769545 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0a7c096a70 0x7f0a7c12c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:09.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.198+0000 7f0a80af8640 1 -- 172.21.15.102:0/2404769545 >> v2:172.21.15.102:3300/0 conn(0x7f0a7c12cd00 msgr2=0x7f0a7c098950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:09.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.198+0000 7f0a80af8640 1 --2- 172.21.15.102:0/2404769545 >> v2:172.21.15.102:3300/0 conn(0x7f0a7c12cd00 0x7f0a7c098950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:09.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.198+0000 7f0a80af8640 1 -- 172.21.15.102:0/2404769545 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a7c09ac30 con 0x7f0a7c12c8b0 2024-09-13T17:44:09.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.198+0000 7f0a81afa640 1 --2- 172.21.15.102:0/2404769545 >> v2:172.21.15.102:3300/0 conn(0x7f0a7c12cd00 0x7f0a7c098950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:09.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.199+0000 7f0a80af8640 1 --2- 172.21.15.102:0/2404769545 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0a7c12c8b0 0x7f0a7c098410 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c00dd40 tx=0x7f0a6c00d660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d9c0b2488d09eba server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:09.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.199+0000 7f0a73fff640 1 -- 172.21.15.102:0/2404769545 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c00d2e0 con 0x7f0a7c12c8b0 2024-09-13T17:44:09.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.199+0000 7f0a73fff640 1 -- 172.21.15.102:0/2404769545 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a6c031050 con 0x7f0a7c12c8b0 2024-09-13T17:44:09.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.199+0000 7f0a73fff640 1 -- 172.21.15.102:0/2404769545 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c00b680 con 0x7f0a7c12c8b0 2024-09-13T17:44:09.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.200+0000 7f0a8355d640 1 -- 172.21.15.102:0/2404769545 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0a7c12c8b0 msgr2=0x7f0a7c098410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:09.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.200+0000 7f0a8355d640 1 --2- 172.21.15.102:0/2404769545 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0a7c12c8b0 0x7f0a7c098410 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c00dd40 tx=0x7f0a6c00d660 comp rx=0 tx=0).stop 2024-09-13T17:44:09.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.200+0000 7f0a8355d640 1 -- 172.21.15.102:0/2404769545 shutdown_connections 2024-09-13T17:44:09.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.200+0000 7f0a8355d640 1 --2- 172.21.15.102:0/2404769545 >> v2:172.21.15.102:3300/0 conn(0x7f0a7c12cd00 0x7f0a7c098950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:09.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.200+0000 7f0a8355d640 1 --2- 172.21.15.102:0/2404769545 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0a7c12c8b0 0x7f0a7c098410 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:09.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.200+0000 7f0a8355d640 1 --2- 172.21.15.102:0/2404769545 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0a7c096a70 0x7f0a7c12c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:09.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.200+0000 7f0a8355d640 1 -- 172.21.15.102:0/2404769545 >> 172.21.15.102:0/2404769545 conn(0x7f0a7c09f2d0 msgr2=0x7f0a7c05db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:09.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.200+0000 7f0a8355d640 1 -- 172.21.15.102:0/2404769545 shutdown_connections 2024-09-13T17:44:09.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.200+0000 7f0a8355d640 1 -- 172.21.15.102:0/2404769545 wait complete. 2024-09-13T17:44:09.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.201+0000 7f0a8355d640 1 Processor -- start 2024-09-13T17:44:09.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.201+0000 7f0a8355d640 1 -- start start 2024-09-13T17:44:09.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.201+0000 7f0a8355d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0a7c096a70 0x7f0a7c129600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:09.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.201+0000 7f0a8355d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a7c12c8b0 0x7f0a7c117700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:09.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.201+0000 7f0a8355d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0a7c117c40 0x7f0a7c118050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:09.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.201+0000 7f0a8355d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c1196f0 con 0x7f0a7c12c8b0 2024-09-13T17:44:09.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.201+0000 7f0a8355d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c11a130 con 0x7f0a7c117c40 2024-09-13T17:44:09.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.201+0000 7f0a8355d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c118590 con 0x7f0a7c096a70 2024-09-13T17:44:09.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.201+0000 7f0a80af8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a7c12c8b0 0x7f0a7c117700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:09.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.201+0000 7f0a812f9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0a7c096a70 0x7f0a7c129600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:09.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.201+0000 7f0a81afa640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0a7c117c40 0x7f0a7c118050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:09.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.201+0000 7f0a80af8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a7c12c8b0 0x7f0a7c117700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34498/0 (socket says 172.21.15.102:34498) 2024-09-13T17:44:09.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.201+0000 7f0a80af8640 1 -- 172.21.15.102:0/2385984456 learned_addr learned my addr 172.21.15.102:0/2385984456 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:09.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.202+0000 7f0a81afa640 1 -- 172.21.15.102:0/2385984456 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0a7c096a70 msgr2=0x7f0a7c129600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:09.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.202+0000 7f0a81afa640 1 --2- 172.21.15.102:0/2385984456 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0a7c096a70 0x7f0a7c129600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:09.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.202+0000 7f0a81afa640 1 -- 172.21.15.102:0/2385984456 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a7c12c8b0 msgr2=0x7f0a7c117700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:09.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.202+0000 7f0a81afa640 1 --2- 172.21.15.102:0/2385984456 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a7c12c8b0 0x7f0a7c117700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:09.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.202+0000 7f0a81afa640 1 -- 172.21.15.102:0/2385984456 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a7400ba40 con 0x7f0a7c117c40 2024-09-13T17:44:09.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.202+0000 7f0a812f9640 1 --2- 172.21.15.102:0/2385984456 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0a7c096a70 0x7f0a7c129600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:09.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.202+0000 7f0a81afa640 1 --2- 172.21.15.102:0/2385984456 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0a7c117c40 0x7f0a7c118050 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0a74009fd0 tx=0x7f0a7400d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:09.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.202+0000 7f0a717fa640 1 -- 172.21.15.102:0/2385984456 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a74010040 con 0x7f0a7c117c40 2024-09-13T17:44:09.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.202+0000 7f0a717fa640 1 -- 172.21.15.102:0/2385984456 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a740163f0 con 0x7f0a7c117c40 2024-09-13T17:44:09.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.202+0000 7f0a8355d640 1 -- 172.21.15.102:0/2385984456 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a6c00d900 con 0x7f0a7c117c40 2024-09-13T17:44:09.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.202+0000 7f0a717fa640 1 -- 172.21.15.102:0/2385984456 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a74016790 con 0x7f0a7c117c40 2024-09-13T17:44:09.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.202+0000 7f0a8355d640 1 -- 172.21.15.102:0/2385984456 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a7c119930 con 0x7f0a7c117c40 2024-09-13T17:44:09.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.203+0000 7f0a717fa640 1 -- 172.21.15.102:0/2385984456 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f0a74003020 con 0x7f0a7c117c40 2024-09-13T17:44:09.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.203+0000 7f0a717fa640 1 -- 172.21.15.102:0/2385984456 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 3660+0+0 (secure 0 0 0) 0x7f0a74042650 con 0x7f0a7c117c40 2024-09-13T17:44:09.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.203+0000 7f0a8355d640 1 -- 172.21.15.102:0/2385984456 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a7c064000 con 0x7f0a7c117c40 2024-09-13T17:44:09.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.207+0000 7f0a717fa640 1 -- 172.21.15.102:0/2385984456 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f0a7401c4b0 con 0x7f0a7c117c40 2024-09-13T17:44:09.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.361+0000 7f0a8355d640 1 -- 172.21.15.102:0/2385984456 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a7c06db00 con 0x7f0a7c117c40 2024-09-13T17:44:09.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.361+0000 7f0a717fa640 1 -- 172.21.15.102:0/2385984456 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+11343 (secure 0 0 0) 0x7f0a74013e60 con 0x7f0a7c117c40 2024-09-13T17:44:09.364 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:44:09.364 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":31,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:44:08.385301+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:44:06.718328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-13T17:44:07.355624+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:44:09.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.363+0000 7f0a8355d640 1 -- 172.21.15.102:0/2385984456 >> v1:172.21.15.102:6816/34937 conn(0x7f0a60031190 legacy=0x7f0a60033640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:09.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.363+0000 7f0a8355d640 1 -- 172.21.15.102:0/2385984456 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0a7c117c40 msgr2=0x7f0a7c118050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:09.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.363+0000 7f0a8355d640 1 --2- 172.21.15.102:0/2385984456 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0a7c117c40 0x7f0a7c118050 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0a74009fd0 tx=0x7f0a7400d040 comp rx=0 tx=0).stop 2024-09-13T17:44:09.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.363+0000 7f0a8355d640 1 -- 172.21.15.102:0/2385984456 shutdown_connections 2024-09-13T17:44:09.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.363+0000 7f0a8355d640 1 --2- 172.21.15.102:0/2385984456 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0a7c117c40 0x7f0a7c118050 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:09.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.363+0000 7f0a8355d640 1 --2- 172.21.15.102:0/2385984456 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a7c12c8b0 0x7f0a7c117700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:09.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.363+0000 7f0a8355d640 1 --2- 172.21.15.102:0/2385984456 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0a7c096a70 0x7f0a7c129600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:09.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.364+0000 7f0a8355d640 1 -- 172.21.15.102:0/2385984456 >> 172.21.15.102:0/2385984456 conn(0x7f0a7c09f2d0 msgr2=0x7f0a7c09d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:09.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.364+0000 7f0a8355d640 1 -- 172.21.15.102:0/2385984456 shutdown_connections 2024-09-13T17:44:09.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.364+0000 7f0a8355d640 1 -- 172.21.15.102:0/2385984456 wait complete. 2024-09-13T17:44:09.376 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:44:09.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.468+0000 7fec25f5d640 1 Processor -- start 2024-09-13T17:44:09.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.469+0000 7fec25f5d640 1 -- start start 2024-09-13T17:44:09.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.469+0000 7fec25f5d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fec20128330 0x7fec20128700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:09.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.469+0000 7fec25f5d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fec2012bf50 0x7fec20133b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:09.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.469+0000 7fec25f5d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec20097be0 con 0x7fec20128c40 2024-09-13T17:44:09.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.469+0000 7fec25f5d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec20098e70 con 0x7fec20128330 2024-09-13T17:44:09.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.469+0000 7fec25f5d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fec20098310 con 0x7fec2012bf50 2024-09-13T17:44:09.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.469+0000 7fec1f7fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fec20128330 0x7fec20128700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:09.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.469+0000 7fec1ffff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fec2012bf50 0x7fec20133b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:09.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.469+0000 7fec1f7fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fec20128330 0x7fec20128700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34508/0 (socket says 172.21.15.102:34508) 2024-09-13T17:44:09.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.469+0000 7fec1ffff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fec2012bf50 0x7fec20133b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54352/0 (socket says 172.21.15.102:54352) 2024-09-13T17:44:09.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.469+0000 7fec1effd640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fec20128c40 0x7fec2012b840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:37478/0 (socket says 172.21.15.102:37478) 2024-09-13T17:44:09.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.469+0000 7fec1f7fe640 1 -- 172.21.15.102:0/631190988 learned_addr learned my addr 172.21.15.102:0/631190988 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:09.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.469+0000 7fec1ffff640 1 -- 172.21.15.102:0/631190988 >> v2:172.21.15.102:3300/0 conn(0x7fec20128330 msgr2=0x7fec20128700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:09.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.469+0000 7fec1ffff640 1 --2- 172.21.15.102:0/631190988 >> v2:172.21.15.102:3300/0 conn(0x7fec20128330 0x7fec20128700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:09.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.469+0000 7fec1ffff640 1 -- 172.21.15.102:0/631190988 >> v1:172.21.15.102:6789/0 conn(0x7fec20128c40 legacy=0x7fec2012b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:09.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.469+0000 7fec1ffff640 1 -- 172.21.15.102:0/631190988 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec200985d0 con 0x7fec2012bf50 2024-09-13T17:44:09.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.470+0000 7fec1ffff640 1 --2- 172.21.15.102:0/631190988 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fec2012bf50 0x7fec20133b90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fec18009e60 tx=0x7fec18009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b52b37517bf1946 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:09.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.470+0000 7fec1e7fc640 1 -- 172.21.15.102:0/631190988 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec18032640 con 0x7fec2012bf50 2024-09-13T17:44:09.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.470+0000 7fec1e7fc640 1 -- 172.21.15.102:0/631190988 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec180327a0 con 0x7fec2012bf50 2024-09-13T17:44:09.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.470+0000 7fec1e7fc640 1 -- 172.21.15.102:0/631190988 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec18032b80 con 0x7fec2012bf50 2024-09-13T17:44:09.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.471+0000 7fec25f5d640 1 -- 172.21.15.102:0/631190988 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fec2012bf50 msgr2=0x7fec20133b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:09.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.471+0000 7fec25f5d640 1 --2- 172.21.15.102:0/631190988 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fec2012bf50 0x7fec20133b90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fec18009e60 tx=0x7fec18009780 comp rx=0 tx=0).stop 2024-09-13T17:44:09.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.471+0000 7fec25f5d640 1 -- 172.21.15.102:0/631190988 shutdown_connections 2024-09-13T17:44:09.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.471+0000 7fec25f5d640 1 --2- 172.21.15.102:0/631190988 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fec2012bf50 0x7fec20133b90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:09.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.471+0000 7fec25f5d640 1 --2- 172.21.15.102:0/631190988 >> v2:172.21.15.102:3300/0 conn(0x7fec20128330 0x7fec20128700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:09.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.471+0000 7fec25f5d640 1 -- 172.21.15.102:0/631190988 >> 172.21.15.102:0/631190988 conn(0x7fec2009e820 msgr2=0x7fec2009ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:09.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.471+0000 7fec25f5d640 1 -- 172.21.15.102:0/631190988 shutdown_connections 2024-09-13T17:44:09.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.471+0000 7fec25f5d640 1 -- 172.21.15.102:0/631190988 wait complete. 2024-09-13T17:44:09.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.472+0000 7fec25f5d640 1 Processor -- start 2024-09-13T17:44:09.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.472+0000 7fec25f5d640 1 -- start start 2024-09-13T17:44:09.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.472+0000 7fec25f5d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fec20128330 0x7fec20114e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:09.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.472+0000 7fec25f5d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fec20128c40 0x7fec201153c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:09.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.472+0000 7fec25f5d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec2012bf50 0x7fec20115900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:09.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.472+0000 7fec25f5d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2009a060 con 0x7fec2012bf50 2024-09-13T17:44:09.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.472+0000 7fec25f5d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec201170a0 con 0x7fec20128330 2024-09-13T17:44:09.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.472+0000 7fec25f5d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fec20117e70 con 0x7fec20128c40 2024-09-13T17:44:09.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.472+0000 7fec1f7fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fec20128330 0x7fec20114e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:09.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.472+0000 7fec1ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec2012bf50 0x7fec20115900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:09.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.472+0000 7fec1effd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fec20128c40 0x7fec201153c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:09.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.472+0000 7fec1f7fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fec20128330 0x7fec20114e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:47056/0 (socket says 172.21.15.102:47056) 2024-09-13T17:44:09.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.472+0000 7fec1f7fe640 1 -- 172.21.15.102:0/592419544 learned_addr learned my addr 172.21.15.102:0/592419544 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:09.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.473+0000 7fec1ffff640 1 -- 172.21.15.102:0/592419544 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fec20128c40 msgr2=0x7fec201153c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:09.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.473+0000 7fec1ffff640 1 --2- 172.21.15.102:0/592419544 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fec20128c40 0x7fec201153c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:09.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.473+0000 7fec1ffff640 1 -- 172.21.15.102:0/592419544 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fec20128330 msgr2=0x7fec20114e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:09.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.473+0000 7fec1ffff640 1 --2- 172.21.15.102:0/592419544 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fec20128330 0x7fec20114e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:09.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.473+0000 7fec1ffff640 1 -- 172.21.15.102:0/592419544 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec18009a20 con 0x7fec2012bf50 2024-09-13T17:44:09.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.473+0000 7fec1effd640 1 --2- 172.21.15.102:0/592419544 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fec20128c40 0x7fec201153c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:09.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.473+0000 7fec1f7fe640 1 --2- 172.21.15.102:0/592419544 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fec20128330 0x7fec20114e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:09.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.473+0000 7fec1ffff640 1 --2- 172.21.15.102:0/592419544 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec2012bf50 0x7fec20115900 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fec18034000 tx=0x7fec1803a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:09.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.473+0000 7fec077fe640 1 -- 172.21.15.102:0/592419544 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec18041070 con 0x7fec2012bf50 2024-09-13T17:44:09.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.473+0000 7fec077fe640 1 -- 172.21.15.102:0/592419544 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1803c8e0 con 0x7fec2012bf50 2024-09-13T17:44:09.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.473+0000 7fec25f5d640 1 -- 172.21.15.102:0/592419544 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec20117360 con 0x7fec2012bf50 2024-09-13T17:44:09.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.473+0000 7fec077fe640 1 -- 172.21.15.102:0/592419544 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1803cca0 con 0x7fec2012bf50 2024-09-13T17:44:09.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.473+0000 7fec25f5d640 1 -- 172.21.15.102:0/592419544 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec20117b40 con 0x7fec2012bf50 2024-09-13T17:44:09.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.474+0000 7fec077fe640 1 -- 172.21.15.102:0/592419544 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fec18003020 con 0x7fec2012bf50 2024-09-13T17:44:09.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.474+0000 7fec077fe640 1 -- 172.21.15.102:0/592419544 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 3660+0+0 (secure 0 0 0) 0x7fec180652c0 con 0x7fec2012bf50 2024-09-13T17:44:09.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.474+0000 7fec25f5d640 1 -- 172.21.15.102:0/592419544 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec20125f00 con 0x7fec2012bf50 2024-09-13T17:44:09.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.479+0000 7fec077fe640 1 -- 172.21.15.102:0/592419544 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fec18036070 con 0x7fec2012bf50 2024-09-13T17:44:09.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.632+0000 7fec25f5d640 1 -- 172.21.15.102:0/592419544 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec20064000 con 0x7fec2012bf50 2024-09-13T17:44:09.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.632+0000 7fec077fe640 1 -- 172.21.15.102:0/592419544 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+11343 (secure 0 0 0) 0x7fec1803ce00 con 0x7fec2012bf50 2024-09-13T17:44:09.635 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:44:09.635 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":31,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:44:08.385301+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:44:06.718328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-13T17:44:07.355624+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:44:09.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.634+0000 7fec25f5d640 1 -- 172.21.15.102:0/592419544 >> v1:172.21.15.102:6816/34937 conn(0x7fec0c031190 legacy=0x7fec0c033640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:09.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.634+0000 7fec25f5d640 1 -- 172.21.15.102:0/592419544 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec2012bf50 msgr2=0x7fec20115900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:09.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.634+0000 7fec25f5d640 1 --2- 172.21.15.102:0/592419544 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec2012bf50 0x7fec20115900 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fec18034000 tx=0x7fec1803a000 comp rx=0 tx=0).stop 2024-09-13T17:44:09.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.634+0000 7fec25f5d640 1 -- 172.21.15.102:0/592419544 shutdown_connections 2024-09-13T17:44:09.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.634+0000 7fec25f5d640 1 --2- 172.21.15.102:0/592419544 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec2012bf50 0x7fec20115900 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:09.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.634+0000 7fec25f5d640 1 --2- 172.21.15.102:0/592419544 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fec20128c40 0x7fec201153c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:09.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.634+0000 7fec25f5d640 1 --2- 172.21.15.102:0/592419544 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fec20128330 0x7fec20114e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:09.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.634+0000 7fec25f5d640 1 -- 172.21.15.102:0/592419544 >> 172.21.15.102:0/592419544 conn(0x7fec2009e820 msgr2=0x7fec20129740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:09.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.634+0000 7fec25f5d640 1 -- 172.21.15.102:0/592419544 shutdown_connections 2024-09-13T17:44:09.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.635+0000 7fec25f5d640 1 -- 172.21.15.102:0/592419544 wait complete. 2024-09-13T17:44:09.648 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-13T17:44:09.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.742+0000 7f7286c4f640 1 Processor -- start 2024-09-13T17:44:09.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.743+0000 7f7286c4f640 1 -- start start 2024-09-13T17:44:09.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.743+0000 7f7286c4f640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f728012f8d0 0x7f728012fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:09.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.743+0000 7f7286c4f640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7280069400 0x7f7280137730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:09.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.743+0000 7f7286c4f640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72800a2840 con 0x7f7280068920 2024-09-13T17:44:09.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.743+0000 7f7286c4f640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7280096d80 con 0x7f728012f8d0 2024-09-13T17:44:09.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.743+0000 7f7286c4f640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72800a0e10 con 0x7f7280069400 2024-09-13T17:44:09.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.743+0000 7f72849eb640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f728012f8d0 0x7f728012fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:09.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.743+0000 7f72851ec640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7280069400 0x7f7280137730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:09.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.743+0000 7f727ffff640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f7280068920 0x7f7280068cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:37492/0 (socket says 172.21.15.102:37492) 2024-09-13T17:44:09.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.743+0000 7f727ffff640 1 -- 172.21.15.102:0/2430448824 learned_addr learned my addr 172.21.15.102:0/2430448824 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:44:09.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.743+0000 7f72851ec640 1 -- 172.21.15.102:0/2430448824 >> v2:172.21.15.102:3300/0 conn(0x7f728012f8d0 msgr2=0x7f728012fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:09.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.743+0000 7f72851ec640 1 --2- 172.21.15.102:0/2430448824 >> v2:172.21.15.102:3300/0 conn(0x7f728012f8d0 0x7f728012fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:09.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.743+0000 7f72851ec640 1 -- 172.21.15.102:0/2430448824 >> v1:172.21.15.102:6789/0 conn(0x7f7280068920 legacy=0x7f7280068cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:09.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.744+0000 7f72851ec640 1 -- 172.21.15.102:0/2430448824 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f728012f6b0 con 0x7f7280069400 2024-09-13T17:44:09.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.744+0000 7f727f7fe640 1 -- 172.21.15.102:0/2430448824 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1352478458 0 0) 0x7f72800a2840 con 0x7f7280068920 2024-09-13T17:44:09.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.744+0000 7f72849eb640 1 --2- 172.21.15.102:0/2430448824 >> v2:172.21.15.102:3300/0 conn(0x7f728012f8d0 0x7f728012fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:09.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.744+0000 7f72851ec640 1 --2- 172.21.15.102:0/2430448824 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7280069400 0x7f7280137730 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7278009e60 tx=0x7f7278009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6600b725da2b574 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:09.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.744+0000 7f727f7fe640 1 -- 172.21.15.102:0/2430448824 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7278007550 con 0x7f7280069400 2024-09-13T17:44:09.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.744+0000 7f727f7fe640 1 -- 172.21.15.102:0/2430448824 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727802d050 con 0x7f7280069400 2024-09-13T17:44:09.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.744+0000 7f727f7fe640 1 -- 172.21.15.102:0/2430448824 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72780078e0 con 0x7f7280069400 2024-09-13T17:44:09.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.745+0000 7f7286c4f640 1 -- 172.21.15.102:0/2430448824 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7280069400 msgr2=0x7f7280137730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:09.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.745+0000 7f7286c4f640 1 --2- 172.21.15.102:0/2430448824 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7280069400 0x7f7280137730 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7278009e60 tx=0x7f7278009780 comp rx=0 tx=0).stop 2024-09-13T17:44:09.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.745+0000 7f7286c4f640 1 -- 172.21.15.102:0/2430448824 shutdown_connections 2024-09-13T17:44:09.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.745+0000 7f7286c4f640 1 --2- 172.21.15.102:0/2430448824 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7280069400 0x7f7280137730 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:09.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.745+0000 7f7286c4f640 1 --2- 172.21.15.102:0/2430448824 >> v2:172.21.15.102:3300/0 conn(0x7f728012f8d0 0x7f728012fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:09.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.745+0000 7f7286c4f640 1 -- 172.21.15.102:0/2430448824 >> 172.21.15.102:0/2430448824 conn(0x7f72800a3420 msgr2=0x7f728006e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:09.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.745+0000 7f7286c4f640 1 -- 172.21.15.102:0/2430448824 shutdown_connections 2024-09-13T17:44:09.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.745+0000 7f7286c4f640 1 -- 172.21.15.102:0/2430448824 wait complete. 2024-09-13T17:44:09.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.746+0000 7f7286c4f640 1 Processor -- start 2024-09-13T17:44:09.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.746+0000 7f7286c4f640 1 -- start start 2024-09-13T17:44:09.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.746+0000 7f7286c4f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7280068920 0x7f7280119060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:09.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.746+0000 7f7286c4f640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7280069400 0x7f72801195a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:09.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.746+0000 7f7286c4f640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f728012f8d0 0x7f7280119ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:09.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.746+0000 7f7286c4f640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728011c1d0 con 0x7f7280068920 2024-09-13T17:44:09.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.746+0000 7f7286c4f640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f728011b990 con 0x7f728012f8d0 2024-09-13T17:44:09.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.746+0000 7f7286c4f640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7280069e80 con 0x7f7280069400 2024-09-13T17:44:09.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.747+0000 7f72849eb640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7280068920 0x7f7280119060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:09.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.747+0000 7f72849eb640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7280068920 0x7f7280119060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34540/0 (socket says 172.21.15.102:34540) 2024-09-13T17:44:09.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.747+0000 7f727ffff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7280069400 0x7f72801195a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:09.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.747+0000 7f72849eb640 1 -- 172.21.15.102:0/485006957 learned_addr learned my addr 172.21.15.102:0/485006957 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:09.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.747+0000 7f72851ec640 1 --2- 172.21.15.102:0/485006957 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f728012f8d0 0x7f7280119ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:09.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.747+0000 7f72849eb640 1 -- 172.21.15.102:0/485006957 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7280069400 msgr2=0x7f72801195a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:09.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.747+0000 7f72849eb640 1 --2- 172.21.15.102:0/485006957 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7280069400 0x7f72801195a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:09.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.747+0000 7f72849eb640 1 -- 172.21.15.102:0/485006957 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f728012f8d0 msgr2=0x7f7280119ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:09.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.747+0000 7f72849eb640 1 --2- 172.21.15.102:0/485006957 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f728012f8d0 0x7f7280119ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:09.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.747+0000 7f72849eb640 1 -- 172.21.15.102:0/485006957 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7274009390 con 0x7f7280068920 2024-09-13T17:44:09.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.747+0000 7f72851ec640 1 --2- 172.21.15.102:0/485006957 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f728012f8d0 0x7f7280119ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:09.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.747+0000 7f727ffff640 1 --2- 172.21.15.102:0/485006957 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7280069400 0x7f72801195a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:09.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.747+0000 7f72849eb640 1 --2- 172.21.15.102:0/485006957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7280068920 0x7f7280119060 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f727400ec80 tx=0x7f72740090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:09.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.747+0000 7f727cff9640 1 -- 172.21.15.102:0/485006957 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727400eec0 con 0x7f7280068920 2024-09-13T17:44:09.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.747+0000 7f727cff9640 1 -- 172.21.15.102:0/485006957 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727400cbc0 con 0x7f7280068920 2024-09-13T17:44:09.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.747+0000 7f7286c4f640 1 -- 172.21.15.102:0/485006957 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7278009a20 con 0x7f7280068920 2024-09-13T17:44:09.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.747+0000 7f727cff9640 1 -- 172.21.15.102:0/485006957 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7274013470 con 0x7f7280068920 2024-09-13T17:44:09.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.747+0000 7f7286c4f640 1 -- 172.21.15.102:0/485006957 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f728011a340 con 0x7f7280068920 2024-09-13T17:44:09.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.748+0000 7f727cff9640 1 -- 172.21.15.102:0/485006957 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f72740137e0 con 0x7f7280068920 2024-09-13T17:44:09.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.748+0000 7f727cff9640 1 -- 172.21.15.102:0/485006957 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 3660+0+0 (secure 0 0 0) 0x7f7274042460 con 0x7f7280068920 2024-09-13T17:44:09.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.748+0000 7f7286c4f640 1 -- 172.21.15.102:0/485006957 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7280064940 con 0x7f7280068920 2024-09-13T17:44:09.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.753+0000 7f727cff9640 1 -- 172.21.15.102:0/485006957 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f7280064940 con 0x7f7280068920 2024-09-13T17:44:09.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:09.936+0000 7f7286c4f640 1 -- 172.21.15.102:0/485006957 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7280064940 con 0x7f7280068920 2024-09-13T17:44:10.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.074+0000 7f727cff9640 1 -- 172.21.15.102:0/485006957 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v10) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7280064940 con 0x7f7280068920 2024-09-13T17:44:10.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.076+0000 7f7286c4f640 1 -- 172.21.15.102:0/485006957 >> v1:172.21.15.102:6816/34937 conn(0x7f7264031160 legacy=0x7f7264033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:10.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.076+0000 7f7286c4f640 1 -- 172.21.15.102:0/485006957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7280068920 msgr2=0x7f7280119060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:10.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.076+0000 7f7286c4f640 1 --2- 172.21.15.102:0/485006957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7280068920 0x7f7280119060 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f727400ec80 tx=0x7f72740090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:10.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.076+0000 7f7286c4f640 1 -- 172.21.15.102:0/485006957 shutdown_connections 2024-09-13T17:44:10.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.076+0000 7f7286c4f640 1 --2- 172.21.15.102:0/485006957 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f728012f8d0 0x7f7280119ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:10.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.076+0000 7f7286c4f640 1 --2- 172.21.15.102:0/485006957 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7280069400 0x7f72801195a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:10.078 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.076+0000 7f7286c4f640 1 --2- 172.21.15.102:0/485006957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7280068920 0x7f7280119060 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:10.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.076+0000 7f7286c4f640 1 -- 172.21.15.102:0/485006957 >> 172.21.15.102:0/485006957 conn(0x7f72800a3420 msgr2=0x7f7280132700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:10.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.076+0000 7f7286c4f640 1 -- 172.21.15.102:0/485006957 shutdown_connections 2024-09-13T17:44:10.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.076+0000 7f7286c4f640 1 -- 172.21.15.102:0/485006957 wait complete. 2024-09-13T17:44:10.088 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-13T17:44:10.088 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-13T17:44:10.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.182+0000 7fc96b19d640 1 Processor -- start 2024-09-13T17:44:10.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.182+0000 7fc96b19d640 1 -- start start 2024-09-13T17:44:10.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.182+0000 7fc96b19d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc964097340 0x7fc964132510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:10.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.182+0000 7fc96b19d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc964097740 0x7fc964134a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:10.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.182+0000 7fc96b19d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc964098040 con 0x7fc96409a620 2024-09-13T17:44:10.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.182+0000 7fc96b19d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96409a9f0 con 0x7fc964097340 2024-09-13T17:44:10.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.182+0000 7fc96b19d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9640982a0 con 0x7fc964097740 2024-09-13T17:44:10.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.182+0000 7fc96973a640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc964097740 0x7fc964134a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:10.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.182+0000 7fc95bfff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc964097340 0x7fc964132510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:10.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.182+0000 7fc96973a640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc964097740 0x7fc964134a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:47086/0 (socket says 172.21.15.102:47086) 2024-09-13T17:44:10.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.182+0000 7fc968f39640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fc96409a620 0x7fc964096c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:37498/0 (socket says 172.21.15.102:37498) 2024-09-13T17:44:10.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.182+0000 7fc96973a640 1 -- 172.21.15.102:0/3106981473 learned_addr learned my addr 172.21.15.102:0/3106981473 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:10.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.183+0000 7fc95bfff640 1 -- 172.21.15.102:0/3106981473 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc964097740 msgr2=0x7fc964134a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:10.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.183+0000 7fc95bfff640 1 --2- 172.21.15.102:0/3106981473 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc964097740 0x7fc964134a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:10.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.183+0000 7fc95bfff640 1 -- 172.21.15.102:0/3106981473 >> v1:172.21.15.102:6789/0 conn(0x7fc96409a620 legacy=0x7fc964096c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:10.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.183+0000 7fc95bfff640 1 -- 172.21.15.102:0/3106981473 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc96409ad50 con 0x7fc964097340 2024-09-13T17:44:10.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.183+0000 7fc95b7fe640 1 -- 172.21.15.102:0/3106981473 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3522328333 0 0) 0x7fc964098040 con 0x7fc96409a620 2024-09-13T17:44:10.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.183+0000 7fc96973a640 1 --2- 172.21.15.102:0/3106981473 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc964097740 0x7fc964134a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:10.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.183+0000 7fc95bfff640 1 --2- 172.21.15.102:0/3106981473 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc964097340 0x7fc964132510 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc954009850 tx=0x7fc9540090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5f45ea8a549662 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:10.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.184+0000 7fc95b7fe640 1 -- 172.21.15.102:0/3106981473 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95402ad00 con 0x7fc964097340 2024-09-13T17:44:10.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.184+0000 7fc95b7fe640 1 -- 172.21.15.102:0/3106981473 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc95402ae60 con 0x7fc964097340 2024-09-13T17:44:10.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.184+0000 7fc95b7fe640 1 -- 172.21.15.102:0/3106981473 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc954032670 con 0x7fc964097340 2024-09-13T17:44:10.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.184+0000 7fc96b19d640 1 -- 172.21.15.102:0/3106981473 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc964097340 msgr2=0x7fc964132510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:10.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.184+0000 7fc96b19d640 1 --2- 172.21.15.102:0/3106981473 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc964097340 0x7fc964132510 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc954009850 tx=0x7fc9540090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:10.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.184+0000 7fc96b19d640 1 -- 172.21.15.102:0/3106981473 shutdown_connections 2024-09-13T17:44:10.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.184+0000 7fc96b19d640 1 --2- 172.21.15.102:0/3106981473 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc964097740 0x7fc964134a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:10.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.184+0000 7fc96b19d640 1 --2- 172.21.15.102:0/3106981473 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc964097340 0x7fc964132510 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:10.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.184+0000 7fc96b19d640 1 -- 172.21.15.102:0/3106981473 >> 172.21.15.102:0/3106981473 conn(0x7fc96409e6b0 msgr2=0x7fc96409eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:10.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.184+0000 7fc96b19d640 1 -- 172.21.15.102:0/3106981473 shutdown_connections 2024-09-13T17:44:10.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.184+0000 7fc96b19d640 1 -- 172.21.15.102:0/3106981473 wait complete. 2024-09-13T17:44:10.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.185+0000 7fc96b19d640 1 Processor -- start 2024-09-13T17:44:10.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.185+0000 7fc96b19d640 1 -- start start 2024-09-13T17:44:10.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.185+0000 7fc96b19d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc964097340 0x7fc964115ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:10.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.185+0000 7fc96b19d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc964097740 0x7fc964116220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:10.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.186+0000 7fc96b19d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc96409a620 0x7fc964116760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:10.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.186+0000 7fc96b19d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc964117d00 con 0x7fc96409a620 2024-09-13T17:44:10.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.186+0000 7fc96b19d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc96411a0a0 con 0x7fc964097340 2024-09-13T17:44:10.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.186+0000 7fc96b19d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9641177c0 con 0x7fc964097740 2024-09-13T17:44:10.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.186+0000 7fc968f39640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc964097340 0x7fc964115ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:10.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.186+0000 7fc968f39640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc964097340 0x7fc964115ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:47092/0 (socket says 172.21.15.102:47092) 2024-09-13T17:44:10.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.186+0000 7fc96973a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc96409a620 0x7fc964116760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:10.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.186+0000 7fc95bfff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc964097740 0x7fc964116220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:10.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.186+0000 7fc968f39640 1 -- 172.21.15.102:0/1965637650 learned_addr learned my addr 172.21.15.102:0/1965637650 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:10.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.186+0000 7fc968f39640 1 -- 172.21.15.102:0/1965637650 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc964097740 msgr2=0x7fc964116220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:10.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.186+0000 7fc968f39640 1 --2- 172.21.15.102:0/1965637650 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc964097740 0x7fc964116220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:10.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.186+0000 7fc968f39640 1 -- 172.21.15.102:0/1965637650 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc96409a620 msgr2=0x7fc964116760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:10.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.186+0000 7fc968f39640 1 --2- 172.21.15.102:0/1965637650 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc96409a620 0x7fc964116760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:10.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.186+0000 7fc968f39640 1 -- 172.21.15.102:0/1965637650 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95c009ae0 con 0x7fc964097340 2024-09-13T17:44:10.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.186+0000 7fc96973a640 1 --2- 172.21.15.102:0/1965637650 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc96409a620 0x7fc964116760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:10.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.186+0000 7fc968f39640 1 --2- 172.21.15.102:0/1965637650 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc964097340 0x7fc964115ce0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc96000c610 tx=0x7fc960007d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:10.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.186+0000 7fc958ff9640 1 -- 172.21.15.102:0/1965637650 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc960007300 con 0x7fc964097340 2024-09-13T17:44:10.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.186+0000 7fc958ff9640 1 -- 172.21.15.102:0/1965637650 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc960007460 con 0x7fc964097340 2024-09-13T17:44:10.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.186+0000 7fc96b19d640 1 -- 172.21.15.102:0/1965637650 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc954009390 con 0x7fc964097340 2024-09-13T17:44:10.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.186+0000 7fc958ff9640 1 -- 172.21.15.102:0/1965637650 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9600047a0 con 0x7fc964097340 2024-09-13T17:44:10.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.186+0000 7fc96b19d640 1 -- 172.21.15.102:0/1965637650 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9641193f0 con 0x7fc964097340 2024-09-13T17:44:10.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.187+0000 7fc958ff9640 1 -- 172.21.15.102:0/1965637650 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fc960004b90 con 0x7fc964097340 2024-09-13T17:44:10.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.187+0000 7fc958ff9640 1 -- 172.21.15.102:0/1965637650 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3660+0+0 (secure 0 0 0) 0x7fc9600410f0 con 0x7fc964097340 2024-09-13T17:44:10.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.187+0000 7fc96b19d640 1 -- 172.21.15.102:0/1965637650 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc964064000 con 0x7fc964097340 2024-09-13T17:44:10.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.192+0000 7fc958ff9640 1 -- 172.21.15.102:0/1965637650 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fc96000f260 con 0x7fc964097340 2024-09-13T17:44:10.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:10.374+0000 7fc96b19d640 1 -- 172.21.15.102:0/1965637650 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc96406e5e0 con 0x7fc964097340 2024-09-13T17:44:11.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.083+0000 7fc958ff9640 1 -- 172.21.15.102:0/1965637650 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v11) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc96000e290 con 0x7fc964097340 2024-09-13T17:44:11.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.085+0000 7fc96b19d640 1 -- 172.21.15.102:0/1965637650 >> v1:172.21.15.102:6816/34937 conn(0x7fc950031160 legacy=0x7fc950033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:11.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.085+0000 7fc96b19d640 1 -- 172.21.15.102:0/1965637650 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc964097340 msgr2=0x7fc964115ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:11.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.085+0000 7fc96b19d640 1 --2- 172.21.15.102:0/1965637650 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc964097340 0x7fc964115ce0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc96000c610 tx=0x7fc960007d00 comp rx=0 tx=0).stop 2024-09-13T17:44:11.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.085+0000 7fc96b19d640 1 -- 172.21.15.102:0/1965637650 shutdown_connections 2024-09-13T17:44:11.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.085+0000 7fc96b19d640 1 --2- 172.21.15.102:0/1965637650 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc96409a620 0x7fc964116760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.085+0000 7fc96b19d640 1 --2- 172.21.15.102:0/1965637650 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc964097740 0x7fc964116220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.085+0000 7fc96b19d640 1 --2- 172.21.15.102:0/1965637650 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc964097340 0x7fc964115ce0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.085+0000 7fc96b19d640 1 -- 172.21.15.102:0/1965637650 >> 172.21.15.102:0/1965637650 conn(0x7fc96409e6b0 msgr2=0x7fc96409d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:11.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.085+0000 7fc96b19d640 1 -- 172.21.15.102:0/1965637650 shutdown_connections 2024-09-13T17:44:11.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.085+0000 7fc96b19d640 1 -- 172.21.15.102:0/1965637650 wait complete. 2024-09-13T17:44:11.101 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-13T17:44:11.102 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:44:11.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.199+0000 7fa74035d640 1 Processor -- start 2024-09-13T17:44:11.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.199+0000 7fa74035d640 1 -- start start 2024-09-13T17:44:11.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.199+0000 7fa74035d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa738128e10 0x7fa738130b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:11.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.199+0000 7fa74035d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa7381310b0 0x7fa738133490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:11.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.199+0000 7fa74035d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa738097be0 con 0x7fa738128330 2024-09-13T17:44:11.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.199+0000 7fa74035d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa738129200 con 0x7fa738128e10 2024-09-13T17:44:11.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.199+0000 7fa74035d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa738098e70 con 0x7fa7381310b0 2024-09-13T17:44:11.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.199+0000 7fa73d8f8640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa738128e10 0x7fa738130b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:11.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.199+0000 7fa73e8fa640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa7381310b0 0x7fa738133490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:11.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.199+0000 7fa73e0f9640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fa738128330 0x7fa738128700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:37504/0 (socket says 172.21.15.102:37504) 2024-09-13T17:44:11.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.199+0000 7fa73e0f9640 1 -- 172.21.15.102:0/459488967 learned_addr learned my addr 172.21.15.102:0/459488967 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:44:11.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.199+0000 7fa73d8f8640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa738128e10 0x7fa738130b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34580/0 (socket says 172.21.15.102:34580) 2024-09-13T17:44:11.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.200+0000 7fa73d8f8640 1 -- 172.21.15.102:0/459488967 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa7381310b0 msgr2=0x7fa738133490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:11.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.200+0000 7fa73d8f8640 1 --2- 172.21.15.102:0/459488967 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa7381310b0 0x7fa738133490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.200+0000 7fa73d8f8640 1 -- 172.21.15.102:0/459488967 >> v1:172.21.15.102:6789/0 conn(0x7fa738128330 legacy=0x7fa738128700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:11.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.200+0000 7fa73d8f8640 1 -- 172.21.15.102:0/459488967 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa738098310 con 0x7fa738128e10 2024-09-13T17:44:11.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.200+0000 7fa73d0f7640 1 -- 172.21.15.102:0/459488967 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3697180339 0 0) 0x7fa738097be0 con 0x7fa738128330 2024-09-13T17:44:11.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.200+0000 7fa73e8fa640 1 --2- 172.21.15.102:0/459488967 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa7381310b0 0x7fa738133490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:11.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.200+0000 7fa73d8f8640 1 --2- 172.21.15.102:0/459488967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa738128e10 0x7fa738130b70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa72800a9f0 tx=0x7fa72800a2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d777d6229b6130a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:11.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.201+0000 7fa73d0f7640 1 -- 172.21.15.102:0/459488967 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa728030510 con 0x7fa738128e10 2024-09-13T17:44:11.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.201+0000 7fa73d0f7640 1 -- 172.21.15.102:0/459488967 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72802e050 con 0x7fa738128e10 2024-09-13T17:44:11.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.201+0000 7fa73d0f7640 1 -- 172.21.15.102:0/459488967 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7280308a0 con 0x7fa738128e10 2024-09-13T17:44:11.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.201+0000 7fa74035d640 1 -- 172.21.15.102:0/459488967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa738128e10 msgr2=0x7fa738130b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:11.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.201+0000 7fa74035d640 1 --2- 172.21.15.102:0/459488967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa738128e10 0x7fa738130b70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa72800a9f0 tx=0x7fa72800a2b0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.201+0000 7fa74035d640 1 -- 172.21.15.102:0/459488967 shutdown_connections 2024-09-13T17:44:11.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.201+0000 7fa74035d640 1 --2- 172.21.15.102:0/459488967 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa7381310b0 0x7fa738133490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.201+0000 7fa74035d640 1 --2- 172.21.15.102:0/459488967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa738128e10 0x7fa738130b70 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.201+0000 7fa74035d640 1 -- 172.21.15.102:0/459488967 >> 172.21.15.102:0/459488967 conn(0x7fa73809e820 msgr2=0x7fa73809ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:11.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.201+0000 7fa74035d640 1 -- 172.21.15.102:0/459488967 shutdown_connections 2024-09-13T17:44:11.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.201+0000 7fa74035d640 1 -- 172.21.15.102:0/459488967 wait complete. 2024-09-13T17:44:11.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.202+0000 7fa74035d640 1 Processor -- start 2024-09-13T17:44:11.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.202+0000 7fa74035d640 1 -- start start 2024-09-13T17:44:11.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.202+0000 7fa74035d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa738128330 0x7fa73812ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:11.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.202+0000 7fa74035d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa738128e10 0x7fa73806af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:11.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.202+0000 7fa74035d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa7381310b0 0x7fa73812f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:11.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.202+0000 7fa74035d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7381301d0 con 0x7fa738128330 2024-09-13T17:44:11.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.202+0000 7fa74035d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7381259c0 con 0x7fa7381310b0 2024-09-13T17:44:11.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.202+0000 7fa74035d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa73812dfd0 con 0x7fa738128e10 2024-09-13T17:44:11.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.203+0000 7fa73e0f9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa738128330 0x7fa73812ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:11.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.203+0000 7fa73e8fa640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa7381310b0 0x7fa73812f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:11.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.203+0000 7fa73e0f9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa738128330 0x7fa73812ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34596/0 (socket says 172.21.15.102:34596) 2024-09-13T17:44:11.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.203+0000 7fa73d8f8640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa738128e10 0x7fa73806af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:11.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.203+0000 7fa73e0f9640 1 -- 172.21.15.102:0/856942123 learned_addr learned my addr 172.21.15.102:0/856942123 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:11.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.203+0000 7fa73e0f9640 1 -- 172.21.15.102:0/856942123 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa738128e10 msgr2=0x7fa73806af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:11.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.203+0000 7fa73e0f9640 1 --2- 172.21.15.102:0/856942123 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa738128e10 0x7fa73806af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.203+0000 7fa73e0f9640 1 -- 172.21.15.102:0/856942123 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa7381310b0 msgr2=0x7fa73812f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:11.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.203+0000 7fa73e0f9640 1 --2- 172.21.15.102:0/856942123 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa7381310b0 0x7fa73812f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.203+0000 7fa73e0f9640 1 -- 172.21.15.102:0/856942123 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa730009a20 con 0x7fa738128330 2024-09-13T17:44:11.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.203+0000 7fa73e8fa640 1 --2- 172.21.15.102:0/856942123 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa7381310b0 0x7fa73812f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:11.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.203+0000 7fa73e0f9640 1 --2- 172.21.15.102:0/856942123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa738128330 0x7fa73812ea70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa73400c5e0 tx=0x7fa734007d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:11.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.203+0000 7fa72e7fc640 1 -- 172.21.15.102:0/856942123 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa734005aa0 con 0x7fa738128330 2024-09-13T17:44:11.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.203+0000 7fa72e7fc640 1 -- 172.21.15.102:0/856942123 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa734005c00 con 0x7fa738128330 2024-09-13T17:44:11.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.203+0000 7fa74035d640 1 -- 172.21.15.102:0/856942123 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa72800a530 con 0x7fa738128330 2024-09-13T17:44:11.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.203+0000 7fa72e7fc640 1 -- 172.21.15.102:0/856942123 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7340070d0 con 0x7fa738128330 2024-09-13T17:44:11.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.203+0000 7fa74035d640 1 -- 172.21.15.102:0/856942123 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa73812e310 con 0x7fa738128330 2024-09-13T17:44:11.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.204+0000 7fa72e7fc640 1 -- 172.21.15.102:0/856942123 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fa7340074c0 con 0x7fa738128330 2024-09-13T17:44:11.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.204+0000 7fa74035d640 1 -- 172.21.15.102:0/856942123 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa738064000 con 0x7fa738128330 2024-09-13T17:44:11.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.204+0000 7fa72e7fc640 1 -- 172.21.15.102:0/856942123 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3660+0+0 (secure 0 0 0) 0x7fa734041420 con 0x7fa738128330 2024-09-13T17:44:11.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.208+0000 7fa72e7fc640 1 -- 172.21.15.102:0/856942123 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fa73401a360 con 0x7fa738128330 2024-09-13T17:44:11.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.389+0000 7fa74035d640 1 -- 172.21.15.102:0/856942123 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa73806bb80 con 0x7fa738128330 2024-09-13T17:44:11.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.390+0000 7fa72e7fc640 1 -- 172.21.15.102:0/856942123 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2268 (secure 0 0 0) 0x7fa734017030 con 0x7fa738128330 2024-09-13T17:44:11.393 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:44:11.393 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:44:08.385597+0000","modified":"2024-09-13T17:44:11.073569+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5078},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5078":{"gid":5078,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6817/2209249660","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":2209249660}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-09-13T17:44:11.394 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 11 2024-09-13T17:44:11.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.392+0000 7fa74035d640 1 -- 172.21.15.102:0/856942123 >> v1:172.21.15.102:6816/34937 conn(0x7fa71c031160 legacy=0x7fa71c033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:11.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.392+0000 7fa74035d640 1 -- 172.21.15.102:0/856942123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa738128330 msgr2=0x7fa73812ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:11.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.392+0000 7fa74035d640 1 --2- 172.21.15.102:0/856942123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa738128330 0x7fa73812ea70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa73400c5e0 tx=0x7fa734007d00 comp rx=0 tx=0).stop 2024-09-13T17:44:11.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.392+0000 7fa74035d640 1 -- 172.21.15.102:0/856942123 shutdown_connections 2024-09-13T17:44:11.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.392+0000 7fa74035d640 1 --2- 172.21.15.102:0/856942123 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa7381310b0 0x7fa73812f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.392+0000 7fa74035d640 1 --2- 172.21.15.102:0/856942123 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa738128e10 0x7fa73806af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.392+0000 7fa74035d640 1 --2- 172.21.15.102:0/856942123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa738128330 0x7fa73812ea70 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.392+0000 7fa74035d640 1 -- 172.21.15.102:0/856942123 >> 172.21.15.102:0/856942123 conn(0x7fa73809e820 msgr2=0x7fa73806e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:11.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.392+0000 7fa74035d640 1 -- 172.21.15.102:0/856942123 shutdown_connections 2024-09-13T17:44:11.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.392+0000 7fa74035d640 1 -- 172.21.15.102:0/856942123 wait complete. 2024-09-13T17:44:11.405 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:44:11.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.496+0000 7fbc036dd640 1 Processor -- start 2024-09-13T17:44:11.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.496+0000 7fbc036dd640 1 -- start start 2024-09-13T17:44:11.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.496+0000 7fbc036dd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbbfc068920 0x7fbbfc09d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:11.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.496+0000 7fbc036dd640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fbbfc09dc50 0x7fbbfc09e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:11.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.496+0000 7fbc036dd640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbfc0695a0 con 0x7fbbfc12f310 2024-09-13T17:44:11.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.496+0000 7fbc036dd640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbfc068cf0 con 0x7fbbfc09dc50 2024-09-13T17:44:11.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.496+0000 7fbc036dd640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc09f090 con 0x7fbbfc068920 2024-09-13T17:44:11.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.496+0000 7fbc00c78640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fbbfc09dc50 0x7fbbfc09e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:11.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.496+0000 7fbc01c7a640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fbbfc12f310 0x7fbbfc12f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:37520/0 (socket says 172.21.15.102:37520) 2024-09-13T17:44:11.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.496+0000 7fbc01479640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbbfc068920 0x7fbbfc09d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:11.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.496+0000 7fbc01c7a640 1 -- 172.21.15.102:0/572156571 learned_addr learned my addr 172.21.15.102:0/572156571 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:44:11.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.496+0000 7fbc01479640 1 -- 172.21.15.102:0/572156571 >> v2:172.21.15.102:3300/0 conn(0x7fbbfc09dc50 msgr2=0x7fbbfc09e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:11.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.496+0000 7fbc01479640 1 --2- 172.21.15.102:0/572156571 >> v2:172.21.15.102:3300/0 conn(0x7fbbfc09dc50 0x7fbbfc09e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.496+0000 7fbc01479640 1 -- 172.21.15.102:0/572156571 >> v1:172.21.15.102:6789/0 conn(0x7fbbfc12f310 legacy=0x7fbbfc12f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:11.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.496+0000 7fbc01479640 1 -- 172.21.15.102:0/572156571 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbfc09cce0 con 0x7fbbfc068920 2024-09-13T17:44:11.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.496+0000 7fbc00c78640 1 --2- 172.21.15.102:0/572156571 >> v2:172.21.15.102:3300/0 conn(0x7fbbfc09dc50 0x7fbbfc09e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:11.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.497+0000 7fbc01479640 1 --2- 172.21.15.102:0/572156571 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbbfc068920 0x7fbbfc09d710 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4009850 tx=0x7fbbf40090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dd40e80c2b8a4f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:11.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.497+0000 7fbbfbfff640 1 -- 172.21.15.102:0/572156571 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf402ad00 con 0x7fbbfc068920 2024-09-13T17:44:11.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.497+0000 7fbbfbfff640 1 -- 172.21.15.102:0/572156571 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf402ae60 con 0x7fbbfc068920 2024-09-13T17:44:11.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.497+0000 7fbbfbfff640 1 -- 172.21.15.102:0/572156571 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4032670 con 0x7fbbfc068920 2024-09-13T17:44:11.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.498+0000 7fbc036dd640 1 -- 172.21.15.102:0/572156571 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbbfc068920 msgr2=0x7fbbfc09d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:11.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.498+0000 7fbc036dd640 1 --2- 172.21.15.102:0/572156571 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbbfc068920 0x7fbbfc09d710 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4009850 tx=0x7fbbf40090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.498+0000 7fbc036dd640 1 -- 172.21.15.102:0/572156571 shutdown_connections 2024-09-13T17:44:11.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.498+0000 7fbc036dd640 1 --2- 172.21.15.102:0/572156571 >> v2:172.21.15.102:3300/0 conn(0x7fbbfc09dc50 0x7fbbfc09e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.498+0000 7fbc036dd640 1 --2- 172.21.15.102:0/572156571 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbbfc068920 0x7fbbfc09d710 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.498+0000 7fbc036dd640 1 -- 172.21.15.102:0/572156571 >> 172.21.15.102:0/572156571 conn(0x7fbbfc06f320 msgr2=0x7fbbfc0a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:11.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.498+0000 7fbc036dd640 1 -- 172.21.15.102:0/572156571 shutdown_connections 2024-09-13T17:44:11.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.498+0000 7fbc036dd640 1 -- 172.21.15.102:0/572156571 wait complete. 2024-09-13T17:44:11.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.499+0000 7fbc036dd640 1 Processor -- start 2024-09-13T17:44:11.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.499+0000 7fbc036dd640 1 -- start start 2024-09-13T17:44:11.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.499+0000 7fbc036dd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbbfc068920 0x7fbbfc11b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:11.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.499+0000 7fbc036dd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbbfc09dc50 0x7fbbfc11b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:11.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.499+0000 7fbc036dd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbbfc12f310 0x7fbbfc13ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:11.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.499+0000 7fbc036dd640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc11add0 con 0x7fbbfc068920 2024-09-13T17:44:11.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.499+0000 7fbc036dd640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc11e020 con 0x7fbbfc09dc50 2024-09-13T17:44:11.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.499+0000 7fbc036dd640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc11cd00 con 0x7fbbfc12f310 2024-09-13T17:44:11.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.499+0000 7fbc01479640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbbfc068920 0x7fbbfc11b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:11.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.499+0000 7fbc00c78640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbbfc09dc50 0x7fbbfc11b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:11.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.499+0000 7fbc01479640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbbfc068920 0x7fbbfc11b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34614/0 (socket says 172.21.15.102:34614) 2024-09-13T17:44:11.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.499+0000 7fbc00c78640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbbfc09dc50 0x7fbbfc11b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:47130/0 (socket says 172.21.15.102:47130) 2024-09-13T17:44:11.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.499+0000 7fbc01479640 1 -- 172.21.15.102:0/3479797929 learned_addr learned my addr 172.21.15.102:0/3479797929 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:11.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.499+0000 7fbc01c7a640 1 --2- 172.21.15.102:0/3479797929 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbbfc12f310 0x7fbbfc13ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:11.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.500+0000 7fbc01479640 1 -- 172.21.15.102:0/3479797929 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbbfc12f310 msgr2=0x7fbbfc13ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:11.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.500+0000 7fbc01479640 1 --2- 172.21.15.102:0/3479797929 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbbfc12f310 0x7fbbfc13ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.500+0000 7fbc01479640 1 -- 172.21.15.102:0/3479797929 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbbfc09dc50 msgr2=0x7fbbfc11b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:11.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.500+0000 7fbc01479640 1 --2- 172.21.15.102:0/3479797929 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbbfc09dc50 0x7fbbfc11b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.500+0000 7fbc01479640 1 -- 172.21.15.102:0/3479797929 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf0009390 con 0x7fbbfc068920 2024-09-13T17:44:11.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.500+0000 7fbc00c78640 1 --2- 172.21.15.102:0/3479797929 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbbfc09dc50 0x7fbbfc11b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:11.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.500+0000 7fbc01479640 1 --2- 172.21.15.102:0/3479797929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbbfc068920 0x7fbbfc11b020 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4009850 tx=0x7fbbf4032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:11.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.500+0000 7fbbf97fa640 1 -- 172.21.15.102:0/3479797929 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4031700 con 0x7fbbfc068920 2024-09-13T17:44:11.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.500+0000 7fbbf97fa640 1 -- 172.21.15.102:0/3479797929 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf4031860 con 0x7fbbfc068920 2024-09-13T17:44:11.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.500+0000 7fbc036dd640 1 -- 172.21.15.102:0/3479797929 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf4009390 con 0x7fbbfc068920 2024-09-13T17:44:11.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.500+0000 7fbbf97fa640 1 -- 172.21.15.102:0/3479797929 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4031c20 con 0x7fbbfc068920 2024-09-13T17:44:11.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.500+0000 7fbc036dd640 1 -- 172.21.15.102:0/3479797929 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbfc11c380 con 0x7fbbfc068920 2024-09-13T17:44:11.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.501+0000 7fbbf97fa640 1 -- 172.21.15.102:0/3479797929 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fbbf4039040 con 0x7fbbfc068920 2024-09-13T17:44:11.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.501+0000 7fbbf97fa640 1 -- 172.21.15.102:0/3479797929 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3660+0+0 (secure 0 0 0) 0x7fbbf4062280 con 0x7fbbfc068920 2024-09-13T17:44:11.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.501+0000 7fbc036dd640 1 -- 172.21.15.102:0/3479797929 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbfc064000 con 0x7fbbfc068920 2024-09-13T17:44:11.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.505+0000 7fbbf97fa640 1 -- 172.21.15.102:0/3479797929 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fbbf4037080 con 0x7fbbfc068920 2024-09-13T17:44:11.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.664+0000 7fbc036dd640 1 -- 172.21.15.102:0/3479797929 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbbfc11dc30 con 0x7fbbfc068920 2024-09-13T17:44:11.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.665+0000 7fbbf97fa640 1 -- 172.21.15.102:0/3479797929 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+11330 (secure 0 0 0) 0x7fbbf4049020 con 0x7fbbfc068920 2024-09-13T17:44:11.671 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:44:11.671 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":32,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:44:10.068600+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:44:06.718328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-13T17:44:07.355624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:44:11.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.667+0000 7fbc036dd640 1 -- 172.21.15.102:0/3479797929 >> v1:172.21.15.102:6816/34937 conn(0x7fbbe8031160 legacy=0x7fbbe8033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:11.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.667+0000 7fbc036dd640 1 -- 172.21.15.102:0/3479797929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbbfc068920 msgr2=0x7fbbfc11b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:11.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.667+0000 7fbc036dd640 1 --2- 172.21.15.102:0/3479797929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbbfc068920 0x7fbbfc11b020 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4009850 tx=0x7fbbf4032000 comp rx=0 tx=0).stop 2024-09-13T17:44:11.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.667+0000 7fbc036dd640 1 -- 172.21.15.102:0/3479797929 shutdown_connections 2024-09-13T17:44:11.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.667+0000 7fbc036dd640 1 --2- 172.21.15.102:0/3479797929 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fbbfc12f310 0x7fbbfc13ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.667+0000 7fbc036dd640 1 --2- 172.21.15.102:0/3479797929 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fbbfc09dc50 0x7fbbfc11b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.667+0000 7fbc036dd640 1 --2- 172.21.15.102:0/3479797929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbbfc068920 0x7fbbfc11b020 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.667+0000 7fbc036dd640 1 -- 172.21.15.102:0/3479797929 >> 172.21.15.102:0/3479797929 conn(0x7fbbfc06f320 msgr2=0x7fbbfc0a1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:11.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.667+0000 7fbc036dd640 1 -- 172.21.15.102:0/3479797929 shutdown_connections 2024-09-13T17:44:11.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.667+0000 7fbc036dd640 1 -- 172.21.15.102:0/3479797929 wait complete. 2024-09-13T17:44:11.681 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-09-13T17:44:11.681 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:44:11.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.771+0000 7f13542af640 1 Processor -- start 2024-09-13T17:44:11.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.772+0000 7f13542af640 1 -- start start 2024-09-13T17:44:11.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.772+0000 7f13542af640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f134c06b140 0x7f134c0998c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:11.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.772+0000 7f13542af640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f134c1325b0 0x7f134c134990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:11.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.772+0000 7f13542af640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f134c06ee90 con 0x7f134c09ac20 2024-09-13T17:44:11.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.772+0000 7f13542af640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f134c06a680 con 0x7f134c06b140 2024-09-13T17:44:11.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.772+0000 7f13542af640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f134c06ac90 con 0x7f134c1325b0 2024-09-13T17:44:11.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.772+0000 7f135284c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f134c1325b0 0x7f134c134990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:11.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.772+0000 7f135204b640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f134c06b140 0x7f134c0998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:11.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.772+0000 7f135284c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f134c1325b0 0x7f134c134990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:47134/0 (socket says 172.21.15.102:47134) 2024-09-13T17:44:11.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.772+0000 7f135184a640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f134c09ac20 0x7f134c099e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:37526/0 (socket says 172.21.15.102:37526) 2024-09-13T17:44:11.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.772+0000 7f135284c640 1 -- 172.21.15.102:0/1931128843 learned_addr learned my addr 172.21.15.102:0/1931128843 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:11.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.773+0000 7f135204b640 1 -- 172.21.15.102:0/1931128843 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f134c1325b0 msgr2=0x7f134c134990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:11.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.773+0000 7f135204b640 1 --2- 172.21.15.102:0/1931128843 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f134c1325b0 0x7f134c134990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.773+0000 7f135204b640 1 -- 172.21.15.102:0/1931128843 >> v1:172.21.15.102:6789/0 conn(0x7f134c09ac20 legacy=0x7f134c099e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:11.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.773+0000 7f135204b640 1 -- 172.21.15.102:0/1931128843 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f134c09a510 con 0x7f134c06b140 2024-09-13T17:44:11.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.773+0000 7f1351049640 1 -- 172.21.15.102:0/1931128843 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2157987865 0 0) 0x7f134c06ee90 con 0x7f134c09ac20 2024-09-13T17:44:11.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.773+0000 7f135284c640 1 --2- 172.21.15.102:0/1931128843 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f134c1325b0 0x7f134c134990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:11.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.773+0000 7f135204b640 1 --2- 172.21.15.102:0/1931128843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f134c06b140 0x7f134c0998c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1348009850 tx=0x7f13480090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c505ad7bdb6db2c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:11.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.773+0000 7f1351049640 1 -- 172.21.15.102:0/1931128843 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134802ac80 con 0x7f134c06b140 2024-09-13T17:44:11.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.773+0000 7f1351049640 1 -- 172.21.15.102:0/1931128843 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f134802ade0 con 0x7f134c06b140 2024-09-13T17:44:11.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.774+0000 7f1351049640 1 -- 172.21.15.102:0/1931128843 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1348032670 con 0x7f134c06b140 2024-09-13T17:44:11.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.774+0000 7f13542af640 1 -- 172.21.15.102:0/1931128843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f134c06b140 msgr2=0x7f134c0998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:11.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.774+0000 7f13542af640 1 --2- 172.21.15.102:0/1931128843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f134c06b140 0x7f134c0998c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1348009850 tx=0x7f13480090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.774+0000 7f13542af640 1 -- 172.21.15.102:0/1931128843 shutdown_connections 2024-09-13T17:44:11.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.774+0000 7f13542af640 1 --2- 172.21.15.102:0/1931128843 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f134c1325b0 0x7f134c134990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.774+0000 7f13542af640 1 --2- 172.21.15.102:0/1931128843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f134c06b140 0x7f134c0998c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.774+0000 7f13542af640 1 -- 172.21.15.102:0/1931128843 >> 172.21.15.102:0/1931128843 conn(0x7f134c09e820 msgr2=0x7f134c09ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:11.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.774+0000 7f13542af640 1 -- 172.21.15.102:0/1931128843 shutdown_connections 2024-09-13T17:44:11.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.774+0000 7f13542af640 1 -- 172.21.15.102:0/1931128843 wait complete. 2024-09-13T17:44:11.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.775+0000 7f13542af640 1 Processor -- start 2024-09-13T17:44:11.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.775+0000 7f13542af640 1 -- start start 2024-09-13T17:44:11.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.775+0000 7f13542af640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f134c06b140 0x7f134c096aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:11.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.775+0000 7f13542af640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f134c09ac20 0x7f134c096fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:11.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.775+0000 7f13542af640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f134c1325b0 0x7f134c097520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:11.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.775+0000 7f13542af640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134c131f30 con 0x7f134c09ac20 2024-09-13T17:44:11.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.775+0000 7f13542af640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f134c11f9b0 con 0x7f134c1325b0 2024-09-13T17:44:11.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.775+0000 7f13542af640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f134c098590 con 0x7f134c06b140 2024-09-13T17:44:11.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.775+0000 7f135184a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f134c09ac20 0x7f134c096fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:11.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.775+0000 7f135204b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f134c06b140 0x7f134c096aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:11.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.775+0000 7f135184a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f134c09ac20 0x7f134c096fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34644/0 (socket says 172.21.15.102:34644) 2024-09-13T17:44:11.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.775+0000 7f135184a640 1 -- 172.21.15.102:0/3380118635 learned_addr learned my addr 172.21.15.102:0/3380118635 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:11.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.775+0000 7f135284c640 1 --2- 172.21.15.102:0/3380118635 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f134c1325b0 0x7f134c097520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:11.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.775+0000 7f135204b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f134c06b140 0x7f134c096aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:54418/0 (socket says 172.21.15.102:54418) 2024-09-13T17:44:11.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.776+0000 7f135284c640 1 -- 172.21.15.102:0/3380118635 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f134c06b140 msgr2=0x7f134c096aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:11.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.776+0000 7f135284c640 1 --2- 172.21.15.102:0/3380118635 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f134c06b140 0x7f134c096aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.776+0000 7f135284c640 1 -- 172.21.15.102:0/3380118635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f134c09ac20 msgr2=0x7f134c096fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:11.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.776+0000 7f135284c640 1 --2- 172.21.15.102:0/3380118635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f134c09ac20 0x7f134c096fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.776+0000 7f135284c640 1 -- 172.21.15.102:0/3380118635 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1344009ae0 con 0x7f134c1325b0 2024-09-13T17:44:11.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.776+0000 7f135184a640 1 --2- 172.21.15.102:0/3380118635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f134c09ac20 0x7f134c096fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:11.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.776+0000 7f135204b640 1 --2- 172.21.15.102:0/3380118635 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f134c06b140 0x7f134c096aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:11.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.776+0000 7f135284c640 1 --2- 172.21.15.102:0/3380118635 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f134c1325b0 0x7f134c097520 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1344009f10 tx=0x7f134400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:11.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.776+0000 7f133e7fc640 1 -- 172.21.15.102:0/3380118635 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1344018070 con 0x7f134c1325b0 2024-09-13T17:44:11.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.776+0000 7f133e7fc640 1 -- 172.21.15.102:0/3380118635 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f134400f040 con 0x7f134c1325b0 2024-09-13T17:44:11.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.776+0000 7f13542af640 1 -- 172.21.15.102:0/3380118635 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1348009390 con 0x7f134c1325b0 2024-09-13T17:44:11.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.776+0000 7f133e7fc640 1 -- 172.21.15.102:0/3380118635 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134400cdd0 con 0x7f134c1325b0 2024-09-13T17:44:11.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.776+0000 7f13542af640 1 -- 172.21.15.102:0/3380118635 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f134c11e2b0 con 0x7f134c1325b0 2024-09-13T17:44:11.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.777+0000 7f133e7fc640 1 -- 172.21.15.102:0/3380118635 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f1344003020 con 0x7f134c1325b0 2024-09-13T17:44:11.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.777+0000 7f133e7fc640 1 -- 172.21.15.102:0/3380118635 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3660+0+0 (secure 0 0 0) 0x7f1344042110 con 0x7f134c1325b0 2024-09-13T17:44:11.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.777+0000 7f13542af640 1 -- 172.21.15.102:0/3380118635 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f134c064000 con 0x7f134c1325b0 2024-09-13T17:44:11.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.781+0000 7f133e7fc640 1 -- 172.21.15.102:0/3380118635 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f1344014e40 con 0x7f134c1325b0 2024-09-13T17:44:11.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.929+0000 7f13542af640 1 -- 172.21.15.102:0/3380118635 --> v1:172.21.15.102:6816/34937 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f134c067690 con 0x7f1330031160 2024-09-13T17:44:11.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.935+0000 7f133e7fc640 1 -- 172.21.15.102:0/3380118635 <== mgr.4101 v1:172.21.15.102:6816/34937 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+276324 (unknown 2965378022 0 401691773) 0x7f134c067690 con 0x7f1330031160 2024-09-13T17:44:11.941 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:44:11.943 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-09-13T17:44:11.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.938+0000 7f13542af640 1 -- 172.21.15.102:0/3380118635 >> v1:172.21.15.102:6816/34937 conn(0x7f1330031160 legacy=0x7f1330033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:11.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.938+0000 7f13542af640 1 -- 172.21.15.102:0/3380118635 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f134c1325b0 msgr2=0x7f134c097520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:11.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.938+0000 7f13542af640 1 --2- 172.21.15.102:0/3380118635 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f134c1325b0 0x7f134c097520 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1344009f10 tx=0x7f134400b040 comp rx=0 tx=0).stop 2024-09-13T17:44:11.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.938+0000 7f13542af640 1 -- 172.21.15.102:0/3380118635 shutdown_connections 2024-09-13T17:44:11.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.938+0000 7f13542af640 1 --2- 172.21.15.102:0/3380118635 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f134c1325b0 0x7f134c097520 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.938+0000 7f13542af640 1 --2- 172.21.15.102:0/3380118635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f134c09ac20 0x7f134c096fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.938+0000 7f13542af640 1 --2- 172.21.15.102:0/3380118635 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f134c06b140 0x7f134c096aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.938+0000 7f13542af640 1 -- 172.21.15.102:0/3380118635 >> 172.21.15.102:0/3380118635 conn(0x7f134c09e820 msgr2=0x7f134c06e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:11.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.938+0000 7f13542af640 1 -- 172.21.15.102:0/3380118635 shutdown_connections 2024-09-13T17:44:11.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:11.938+0000 7f13542af640 1 -- 172.21.15.102:0/3380118635 wait complete. 2024-09-13T17:44:11.952 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2024-09-13T17:44:10.440476+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592725,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":125,"num_write_kb":1121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":93,"ondisk_log_size":93,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":93,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1191648,"kb_used_data":2212,"kb_used_omap":6,"kb_used_meta":1189369,"kb_avail":376295712,"statfs":{"total":386547056640,"available":385326809088,"internally_reserved":0,"allocated":2265088,"data_stored":1561458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6816,"internal_metadata":1217914208},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000814"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.096521+0000","last_change":"2024-09-13T17:44:09.096521+0000","last_active":"2024-09-13T17:44:09.096521+0000","last_peered":"2024-09-13T17:44:09.096521+0000","last_clean":"2024-09-13T17:44:09.096521+0000","last_became_active":"2024-09-13T17:44:09.096289+0000","last_became_peered":"2024-09-13T17:44:09.096289+0000","last_unstale":"2024-09-13T17:44:09.096521+0000","last_undegraded":"2024-09-13T17:44:09.096521+0000","last_fullsized":"2024-09-13T17:44:09.096521+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:21.494093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.100887+0000","last_change":"2024-09-13T17:44:09.100887+0000","last_active":"2024-09-13T17:44:09.100887+0000","last_peered":"2024-09-13T17:44:09.100887+0000","last_clean":"2024-09-13T17:44:09.100887+0000","last_became_active":"2024-09-13T17:44:09.100407+0000","last_became_peered":"2024-09-13T17:44:09.100407+0000","last_unstale":"2024-09-13T17:44:09.100887+0000","last_undegraded":"2024-09-13T17:44:09.100887+0000","last_fullsized":"2024-09-13T17:44:09.100887+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:00:01.667786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.084879+0000","last_change":"2024-09-13T17:44:09.084879+0000","last_active":"2024-09-13T17:44:09.084879+0000","last_peered":"2024-09-13T17:44:09.084879+0000","last_clean":"2024-09-13T17:44:09.084879+0000","last_became_active":"2024-09-13T17:44:09.084643+0000","last_became_peered":"2024-09-13T17:44:09.084643+0000","last_unstale":"2024-09-13T17:44:09.084879+0000","last_undegraded":"2024-09-13T17:44:09.084879+0000","last_fullsized":"2024-09-13T17:44:09.084879+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:41:47.399207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.105480+0000","last_change":"2024-09-13T17:44:09.105480+0000","last_active":"2024-09-13T17:44:09.105480+0000","last_peered":"2024-09-13T17:44:09.105480+0000","last_clean":"2024-09-13T17:44:09.105480+0000","last_became_active":"2024-09-13T17:44:09.105242+0000","last_became_peered":"2024-09-13T17:44:09.105242+0000","last_unstale":"2024-09-13T17:44:09.105480+0000","last_undegraded":"2024-09-13T17:44:09.105480+0000","last_fullsized":"2024-09-13T17:44:09.105480+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:17:45.470130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.096825+0000","last_change":"2024-09-13T17:44:09.096825+0000","last_active":"2024-09-13T17:44:09.096825+0000","last_peered":"2024-09-13T17:44:09.096825+0000","last_clean":"2024-09-13T17:44:09.096825+0000","last_became_active":"2024-09-13T17:44:09.096589+0000","last_became_peered":"2024-09-13T17:44:09.096589+0000","last_unstale":"2024-09-13T17:44:09.096825+0000","last_undegraded":"2024-09-13T17:44:09.096825+0000","last_fullsized":"2024-09-13T17:44:09.096825+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:46.797545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.082737+0000","last_change":"2024-09-13T17:44:09.082737+0000","last_active":"2024-09-13T17:44:09.082737+0000","last_peered":"2024-09-13T17:44:09.082737+0000","last_clean":"2024-09-13T17:44:09.082737+0000","last_became_active":"2024-09-13T17:44:09.082447+0000","last_became_peered":"2024-09-13T17:44:09.082447+0000","last_unstale":"2024-09-13T17:44:09.082737+0000","last_undegraded":"2024-09-13T17:44:09.082737+0000","last_fullsized":"2024-09-13T17:44:09.082737+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:18:08.511918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.076491+0000","last_change":"2024-09-13T17:44:09.076491+0000","last_active":"2024-09-13T17:44:09.076491+0000","last_peered":"2024-09-13T17:44:09.076491+0000","last_clean":"2024-09-13T17:44:09.076491+0000","last_became_active":"2024-09-13T17:44:09.076262+0000","last_became_peered":"2024-09-13T17:44:09.076262+0000","last_unstale":"2024-09-13T17:44:09.076491+0000","last_undegraded":"2024-09-13T17:44:09.076491+0000","last_fullsized":"2024-09-13T17:44:09.076491+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:46.797545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.081612+0000","last_change":"2024-09-13T17:44:09.081612+0000","last_active":"2024-09-13T17:44:09.081612+0000","last_peered":"2024-09-13T17:44:09.081612+0000","last_clean":"2024-09-13T17:44:09.081612+0000","last_became_active":"2024-09-13T17:44:09.081341+0000","last_became_peered":"2024-09-13T17:44:09.081341+0000","last_unstale":"2024-09-13T17:44:09.081612+0000","last_undegraded":"2024-09-13T17:44:09.081612+0000","last_fullsized":"2024-09-13T17:44:09.081612+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:19:45.314545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.083298+0000","last_change":"2024-09-13T17:44:09.083298+0000","last_active":"2024-09-13T17:44:09.083298+0000","last_peered":"2024-09-13T17:44:09.083298+0000","last_clean":"2024-09-13T17:44:09.083298+0000","last_became_active":"2024-09-13T17:44:09.083029+0000","last_became_peered":"2024-09-13T17:44:09.083029+0000","last_unstale":"2024-09-13T17:44:09.083298+0000","last_undegraded":"2024-09-13T17:44:09.083298+0000","last_fullsized":"2024-09-13T17:44:09.083298+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:48.852877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.069341+0000","last_change":"2024-09-13T17:44:09.069341+0000","last_active":"2024-09-13T17:44:09.069341+0000","last_peered":"2024-09-13T17:44:09.069341+0000","last_clean":"2024-09-13T17:44:09.069341+0000","last_became_active":"2024-09-13T17:44:09.069095+0000","last_became_peered":"2024-09-13T17:44:09.069095+0000","last_unstale":"2024-09-13T17:44:09.069341+0000","last_undegraded":"2024-09-13T17:44:09.069341+0000","last_fullsized":"2024-09-13T17:44:09.069341+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:40:06.063189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.078841+0000","last_change":"2024-09-13T17:44:09.078841+0000","last_active":"2024-09-13T17:44:09.078841+0000","last_peered":"2024-09-13T17:44:09.078841+0000","last_clean":"2024-09-13T17:44:09.078841+0000","last_became_active":"2024-09-13T17:44:09.078254+0000","last_became_peered":"2024-09-13T17:44:09.078254+0000","last_unstale":"2024-09-13T17:44:09.078841+0000","last_undegraded":"2024-09-13T17:44:09.078841+0000","last_fullsized":"2024-09-13T17:44:09.078841+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:23:33.695580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.085491+0000","last_change":"2024-09-13T17:44:09.085491+0000","last_active":"2024-09-13T17:44:09.085491+0000","last_peered":"2024-09-13T17:44:09.085491+0000","last_clean":"2024-09-13T17:44:09.085491+0000","last_became_active":"2024-09-13T17:44:09.085172+0000","last_became_peered":"2024-09-13T17:44:09.085172+0000","last_unstale":"2024-09-13T17:44:09.085491+0000","last_undegraded":"2024-09-13T17:44:09.085491+0000","last_fullsized":"2024-09-13T17:44:09.085491+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:40:06.063189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.087234+0000","last_change":"2024-09-13T17:44:09.087234+0000","last_active":"2024-09-13T17:44:09.087234+0000","last_peered":"2024-09-13T17:44:09.087234+0000","last_clean":"2024-09-13T17:44:09.087234+0000","last_became_active":"2024-09-13T17:44:09.086891+0000","last_became_peered":"2024-09-13T17:44:09.086891+0000","last_unstale":"2024-09-13T17:44:09.087234+0000","last_undegraded":"2024-09-13T17:44:09.087234+0000","last_fullsized":"2024-09-13T17:44:09.087234+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:38.841320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.100794+0000","last_change":"2024-09-13T17:44:09.100794+0000","last_active":"2024-09-13T17:44:09.100794+0000","last_peered":"2024-09-13T17:44:09.100794+0000","last_clean":"2024-09-13T17:44:09.100794+0000","last_became_active":"2024-09-13T17:44:09.100575+0000","last_became_peered":"2024-09-13T17:44:09.100575+0000","last_unstale":"2024-09-13T17:44:09.100794+0000","last_undegraded":"2024-09-13T17:44:09.100794+0000","last_fullsized":"2024-09-13T17:44:09.100794+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:38.841320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.101130+0000","last_change":"2024-09-13T17:44:09.101130+0000","last_active":"2024-09-13T17:44:09.101130+0000","last_peered":"2024-09-13T17:44:09.101130+0000","last_clean":"2024-09-13T17:44:09.101130+0000","last_became_active":"2024-09-13T17:44:09.100865+0000","last_became_peered":"2024-09-13T17:44:09.100865+0000","last_unstale":"2024-09-13T17:44:09.101130+0000","last_undegraded":"2024-09-13T17:44:09.101130+0000","last_fullsized":"2024-09-13T17:44:09.101130+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:53:41.289117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.076274+0000","last_change":"2024-09-13T17:44:09.076274+0000","last_active":"2024-09-13T17:44:09.076274+0000","last_peered":"2024-09-13T17:44:09.076274+0000","last_clean":"2024-09-13T17:44:09.076274+0000","last_became_active":"2024-09-13T17:44:09.075782+0000","last_became_peered":"2024-09-13T17:44:09.075782+0000","last_unstale":"2024-09-13T17:44:09.076274+0000","last_undegraded":"2024-09-13T17:44:09.076274+0000","last_fullsized":"2024-09-13T17:44:09.076274+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:23:33.695580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.095089+0000","last_change":"2024-09-13T17:44:09.095089+0000","last_active":"2024-09-13T17:44:09.095089+0000","last_peered":"2024-09-13T17:44:09.095089+0000","last_clean":"2024-09-13T17:44:09.095089+0000","last_became_active":"2024-09-13T17:44:09.094856+0000","last_became_peered":"2024-09-13T17:44:09.094856+0000","last_unstale":"2024-09-13T17:44:09.095089+0000","last_undegraded":"2024-09-13T17:44:09.095089+0000","last_fullsized":"2024-09-13T17:44:09.095089+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:52.363453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.095140+0000","last_change":"2024-09-13T17:44:09.095140+0000","last_active":"2024-09-13T17:44:09.095140+0000","last_peered":"2024-09-13T17:44:09.095140+0000","last_clean":"2024-09-13T17:44:09.095140+0000","last_became_active":"2024-09-13T17:44:09.094922+0000","last_became_peered":"2024-09-13T17:44:09.094922+0000","last_unstale":"2024-09-13T17:44:09.095140+0000","last_undegraded":"2024-09-13T17:44:09.095140+0000","last_fullsized":"2024-09-13T17:44:09.095140+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:07.163678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.097429+0000","last_change":"2024-09-13T17:44:09.097429+0000","last_active":"2024-09-13T17:44:09.097429+0000","last_peered":"2024-09-13T17:44:09.097429+0000","last_clean":"2024-09-13T17:44:09.097429+0000","last_became_active":"2024-09-13T17:44:09.097178+0000","last_became_peered":"2024-09-13T17:44:09.097178+0000","last_unstale":"2024-09-13T17:44:09.097429+0000","last_undegraded":"2024-09-13T17:44:09.097429+0000","last_fullsized":"2024-09-13T17:44:09.097429+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:30:37.262970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.083241+0000","last_change":"2024-09-13T17:44:09.083241+0000","last_active":"2024-09-13T17:44:09.083241+0000","last_peered":"2024-09-13T17:44:09.083241+0000","last_clean":"2024-09-13T17:44:09.083241+0000","last_became_active":"2024-09-13T17:44:09.082984+0000","last_became_peered":"2024-09-13T17:44:09.082984+0000","last_unstale":"2024-09-13T17:44:09.083241+0000","last_undegraded":"2024-09-13T17:44:09.083241+0000","last_fullsized":"2024-09-13T17:44:09.083241+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:00:07.720440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.077433+0000","last_change":"2024-09-13T17:44:09.077433+0000","last_active":"2024-09-13T17:44:09.077433+0000","last_peered":"2024-09-13T17:44:09.077433+0000","last_clean":"2024-09-13T17:44:09.077433+0000","last_became_active":"2024-09-13T17:44:09.076107+0000","last_became_peered":"2024-09-13T17:44:09.076107+0000","last_unstale":"2024-09-13T17:44:09.077433+0000","last_undegraded":"2024-09-13T17:44:09.077433+0000","last_fullsized":"2024-09-13T17:44:09.077433+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:10:25.431639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.086733+0000","last_change":"2024-09-13T17:44:09.086733+0000","last_active":"2024-09-13T17:44:09.086733+0000","last_peered":"2024-09-13T17:44:09.086733+0000","last_clean":"2024-09-13T17:44:09.086733+0000","last_became_active":"2024-09-13T17:44:09.086214+0000","last_became_peered":"2024-09-13T17:44:09.086214+0000","last_unstale":"2024-09-13T17:44:09.086733+0000","last_undegraded":"2024-09-13T17:44:09.086733+0000","last_fullsized":"2024-09-13T17:44:09.086733+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:29.109889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.100766+0000","last_change":"2024-09-13T17:44:09.100766+0000","last_active":"2024-09-13T17:44:09.100766+0000","last_peered":"2024-09-13T17:44:09.100766+0000","last_clean":"2024-09-13T17:44:09.100766+0000","last_became_active":"2024-09-13T17:44:09.100532+0000","last_became_peered":"2024-09-13T17:44:09.100532+0000","last_unstale":"2024-09-13T17:44:09.100766+0000","last_undegraded":"2024-09-13T17:44:09.100766+0000","last_fullsized":"2024-09-13T17:44:09.100766+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:23:20.557832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.082870+0000","last_change":"2024-09-13T17:44:09.082870+0000","last_active":"2024-09-13T17:44:09.082870+0000","last_peered":"2024-09-13T17:44:09.082870+0000","last_clean":"2024-09-13T17:44:09.082870+0000","last_became_active":"2024-09-13T17:44:09.082641+0000","last_became_peered":"2024-09-13T17:44:09.082641+0000","last_unstale":"2024-09-13T17:44:09.082870+0000","last_undegraded":"2024-09-13T17:44:09.082870+0000","last_fullsized":"2024-09-13T17:44:09.082870+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:23:36.075940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.097350+0000","last_change":"2024-09-13T17:44:09.097350+0000","last_active":"2024-09-13T17:44:09.097350+0000","last_peered":"2024-09-13T17:44:09.097350+0000","last_clean":"2024-09-13T17:44:09.097350+0000","last_became_active":"2024-09-13T17:44:09.097114+0000","last_became_peered":"2024-09-13T17:44:09.097114+0000","last_unstale":"2024-09-13T17:44:09.097350+0000","last_undegraded":"2024-09-13T17:44:09.097350+0000","last_fullsized":"2024-09-13T17:44:09.097350+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:10:25.431639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.085005+0000","last_change":"2024-09-13T17:44:09.085005+0000","last_active":"2024-09-13T17:44:09.085005+0000","last_peered":"2024-09-13T17:44:09.085005+0000","last_clean":"2024-09-13T17:44:09.085005+0000","last_became_active":"2024-09-13T17:44:09.084503+0000","last_became_peered":"2024-09-13T17:44:09.084503+0000","last_unstale":"2024-09-13T17:44:09.085005+0000","last_undegraded":"2024-09-13T17:44:09.085005+0000","last_fullsized":"2024-09-13T17:44:09.085005+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:30:37.262970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.085567+0000","last_change":"2024-09-13T17:44:09.085567+0000","last_active":"2024-09-13T17:44:09.085567+0000","last_peered":"2024-09-13T17:44:09.085567+0000","last_clean":"2024-09-13T17:44:09.085567+0000","last_became_active":"2024-09-13T17:44:09.085234+0000","last_became_peered":"2024-09-13T17:44:09.085234+0000","last_unstale":"2024-09-13T17:44:09.085567+0000","last_undegraded":"2024-09-13T17:44:09.085567+0000","last_fullsized":"2024-09-13T17:44:09.085567+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:43.983384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.082703+0000","last_change":"2024-09-13T17:44:09.082703+0000","last_active":"2024-09-13T17:44:09.082703+0000","last_peered":"2024-09-13T17:44:09.082703+0000","last_clean":"2024-09-13T17:44:09.082703+0000","last_became_active":"2024-09-13T17:44:09.082474+0000","last_became_peered":"2024-09-13T17:44:09.082474+0000","last_unstale":"2024-09-13T17:44:09.082703+0000","last_undegraded":"2024-09-13T17:44:09.082703+0000","last_fullsized":"2024-09-13T17:44:09.082703+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:44.944657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.082198+0000","last_change":"2024-09-13T17:44:09.082198+0000","last_active":"2024-09-13T17:44:09.082198+0000","last_peered":"2024-09-13T17:44:09.082198+0000","last_clean":"2024-09-13T17:44:09.082198+0000","last_became_active":"2024-09-13T17:44:09.081672+0000","last_became_peered":"2024-09-13T17:44:09.081672+0000","last_unstale":"2024-09-13T17:44:09.082198+0000","last_undegraded":"2024-09-13T17:44:09.082198+0000","last_fullsized":"2024-09-13T17:44:09.082198+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:15:17.170944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.084099+0000","last_change":"2024-09-13T17:44:09.084099+0000","last_active":"2024-09-13T17:44:09.084099+0000","last_peered":"2024-09-13T17:44:09.084099+0000","last_clean":"2024-09-13T17:44:09.084099+0000","last_became_active":"2024-09-13T17:44:09.082155+0000","last_became_peered":"2024-09-13T17:44:09.082155+0000","last_unstale":"2024-09-13T17:44:09.084099+0000","last_undegraded":"2024-09-13T17:44:09.084099+0000","last_fullsized":"2024-09-13T17:44:09.084099+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:08:37.816520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.086616+0000","last_change":"2024-09-13T17:44:09.086616+0000","last_active":"2024-09-13T17:44:09.086616+0000","last_peered":"2024-09-13T17:44:09.086616+0000","last_clean":"2024-09-13T17:44:09.086616+0000","last_became_active":"2024-09-13T17:44:09.085504+0000","last_became_peered":"2024-09-13T17:44:09.085504+0000","last_unstale":"2024-09-13T17:44:09.086616+0000","last_undegraded":"2024-09-13T17:44:09.086616+0000","last_fullsized":"2024-09-13T17:44:09.086616+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:43.983384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.085649+0000","last_change":"2024-09-13T17:44:09.085649+0000","last_active":"2024-09-13T17:44:09.085649+0000","last_peered":"2024-09-13T17:44:09.085649+0000","last_clean":"2024-09-13T17:44:09.085649+0000","last_became_active":"2024-09-13T17:44:09.085422+0000","last_became_peered":"2024-09-13T17:44:09.085422+0000","last_unstale":"2024-09-13T17:44:09.085649+0000","last_undegraded":"2024-09-13T17:44:09.085649+0000","last_fullsized":"2024-09-13T17:44:09.085649+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:17:45.470130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.090868+0000","last_change":"2024-09-13T17:44:09.090868+0000","last_active":"2024-09-13T17:44:09.090868+0000","last_peered":"2024-09-13T17:44:09.090868+0000","last_clean":"2024-09-13T17:44:09.090868+0000","last_became_active":"2024-09-13T17:44:09.090651+0000","last_became_peered":"2024-09-13T17:44:09.090651+0000","last_unstale":"2024-09-13T17:44:09.090868+0000","last_undegraded":"2024-09-13T17:44:09.090868+0000","last_fullsized":"2024-09-13T17:44:09.090868+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:59:17.503064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"31'4","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.103927+0000","last_change":"2024-09-13T17:44:08.078924+0000","last_active":"2024-09-13T17:44:09.103927+0000","last_peered":"2024-09-13T17:44:09.103927+0000","last_clean":"2024-09-13T17:44:09.103927+0000","last_became_active":"2024-09-13T17:44:08.078686+0000","last_became_peered":"2024-09-13T17:44:08.078686+0000","last_unstale":"2024-09-13T17:44:09.103927+0000","last_undegraded":"2024-09-13T17:44:09.103927+0000","last_fullsized":"2024-09-13T17:44:09.103927+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:00:18.297348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.082059+0000","last_change":"2024-09-13T17:44:09.082059+0000","last_active":"2024-09-13T17:44:09.082059+0000","last_peered":"2024-09-13T17:44:09.082059+0000","last_clean":"2024-09-13T17:44:09.082059+0000","last_became_active":"2024-09-13T17:44:09.080468+0000","last_became_peered":"2024-09-13T17:44:09.080468+0000","last_unstale":"2024-09-13T17:44:09.082059+0000","last_undegraded":"2024-09-13T17:44:09.082059+0000","last_fullsized":"2024-09-13T17:44:09.082059+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:00:07.720440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"31'1","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.066141+0000","last_change":"2024-09-13T17:44:08.087199+0000","last_active":"2024-09-13T17:44:09.066141+0000","last_peered":"2024-09-13T17:44:09.066141+0000","last_clean":"2024-09-13T17:44:09.066141+0000","last_became_active":"2024-09-13T17:44:08.086469+0000","last_became_peered":"2024-09-13T17:44:08.086469+0000","last_unstale":"2024-09-13T17:44:09.066141+0000","last_undegraded":"2024-09-13T17:44:09.066141+0000","last_fullsized":"2024-09-13T17:44:09.066141+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:45.333857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.100981+0000","last_change":"2024-09-13T17:44:09.100981+0000","last_active":"2024-09-13T17:44:09.100981+0000","last_peered":"2024-09-13T17:44:09.100981+0000","last_clean":"2024-09-13T17:44:09.100981+0000","last_became_active":"2024-09-13T17:44:09.100713+0000","last_became_peered":"2024-09-13T17:44:09.100713+0000","last_unstale":"2024-09-13T17:44:09.100981+0000","last_undegraded":"2024-09-13T17:44:09.100981+0000","last_fullsized":"2024-09-13T17:44:09.100981+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:30:59.005533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.094766+0000","last_change":"2024-09-13T17:44:08.085058+0000","last_active":"2024-09-13T17:44:09.094766+0000","last_peered":"2024-09-13T17:44:09.094766+0000","last_clean":"2024-09-13T17:44:09.094766+0000","last_became_active":"2024-09-13T17:44:08.084831+0000","last_became_peered":"2024-09-13T17:44:08.084831+0000","last_unstale":"2024-09-13T17:44:09.094766+0000","last_undegraded":"2024-09-13T17:44:09.094766+0000","last_fullsized":"2024-09-13T17:44:09.094766+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:31.939247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.087821+0000","last_change":"2024-09-13T17:44:09.087821+0000","last_active":"2024-09-13T17:44:09.087821+0000","last_peered":"2024-09-13T17:44:09.087821+0000","last_clean":"2024-09-13T17:44:09.087821+0000","last_became_active":"2024-09-13T17:44:09.087579+0000","last_became_peered":"2024-09-13T17:44:09.087579+0000","last_unstale":"2024-09-13T17:44:09.087821+0000","last_undegraded":"2024-09-13T17:44:09.087821+0000","last_fullsized":"2024-09-13T17:44:09.087821+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:03.454864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.106047+0000","last_change":"2024-09-13T17:44:08.079732+0000","last_active":"2024-09-13T17:44:09.106047+0000","last_peered":"2024-09-13T17:44:09.106047+0000","last_clean":"2024-09-13T17:44:09.106047+0000","last_became_active":"2024-09-13T17:44:08.079375+0000","last_became_peered":"2024-09-13T17:44:08.079375+0000","last_unstale":"2024-09-13T17:44:09.106047+0000","last_undegraded":"2024-09-13T17:44:09.106047+0000","last_fullsized":"2024-09-13T17:44:09.106047+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:16:25.243221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.078321+0000","last_change":"2024-09-13T17:44:09.078321+0000","last_active":"2024-09-13T17:44:09.078321+0000","last_peered":"2024-09-13T17:44:09.078321+0000","last_clean":"2024-09-13T17:44:09.078321+0000","last_became_active":"2024-09-13T17:44:09.078085+0000","last_became_peered":"2024-09-13T17:44:09.078085+0000","last_unstale":"2024-09-13T17:44:09.078321+0000","last_undegraded":"2024-09-13T17:44:09.078321+0000","last_fullsized":"2024-09-13T17:44:09.078321+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:08:00.966936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"31'1","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.074489+0000","last_change":"2024-09-13T17:44:08.087304+0000","last_active":"2024-09-13T17:44:09.074489+0000","last_peered":"2024-09-13T17:44:09.074489+0000","last_clean":"2024-09-13T17:44:09.074489+0000","last_became_active":"2024-09-13T17:44:08.086913+0000","last_became_peered":"2024-09-13T17:44:08.086913+0000","last_unstale":"2024-09-13T17:44:09.074489+0000","last_undegraded":"2024-09-13T17:44:09.074489+0000","last_fullsized":"2024-09-13T17:44:09.074489+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:06:31.498523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.078727+0000","last_change":"2024-09-13T17:44:09.078727+0000","last_active":"2024-09-13T17:44:09.078727+0000","last_peered":"2024-09-13T17:44:09.078727+0000","last_clean":"2024-09-13T17:44:09.078727+0000","last_became_active":"2024-09-13T17:44:09.078337+0000","last_became_peered":"2024-09-13T17:44:09.078337+0000","last_unstale":"2024-09-13T17:44:09.078727+0000","last_undegraded":"2024-09-13T17:44:09.078727+0000","last_fullsized":"2024-09-13T17:44:09.078727+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:30:37.262970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"31'1","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.062588+0000","last_change":"2024-09-13T17:44:08.088763+0000","last_active":"2024-09-13T17:44:09.062588+0000","last_peered":"2024-09-13T17:44:09.062588+0000","last_clean":"2024-09-13T17:44:09.062588+0000","last_became_active":"2024-09-13T17:44:08.088465+0000","last_became_peered":"2024-09-13T17:44:08.088465+0000","last_unstale":"2024-09-13T17:44:09.062588+0000","last_undegraded":"2024-09-13T17:44:09.062588+0000","last_fullsized":"2024-09-13T17:44:09.062588+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:45.333857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.086254+0000","last_change":"2024-09-13T17:44:09.086254+0000","last_active":"2024-09-13T17:44:09.086254+0000","last_peered":"2024-09-13T17:44:09.086254+0000","last_clean":"2024-09-13T17:44:09.086254+0000","last_became_active":"2024-09-13T17:44:09.084768+0000","last_became_peered":"2024-09-13T17:44:09.084768+0000","last_unstale":"2024-09-13T17:44:09.086254+0000","last_undegraded":"2024-09-13T17:44:09.086254+0000","last_fullsized":"2024-09-13T17:44:09.086254+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:34:38.550964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.089244+0000","last_change":"2024-09-13T17:44:08.080175+0000","last_active":"2024-09-13T17:44:08.089244+0000","last_peered":"2024-09-13T17:44:08.089244+0000","last_clean":"2024-09-13T17:44:08.089244+0000","last_became_active":"2024-09-13T17:44:08.079917+0000","last_became_peered":"2024-09-13T17:44:08.079917+0000","last_unstale":"2024-09-13T17:44:08.089244+0000","last_undegraded":"2024-09-13T17:44:08.089244+0000","last_fullsized":"2024-09-13T17:44:08.089244+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:11:10.137498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.092808+0000","last_change":"2024-09-13T17:44:09.092808+0000","last_active":"2024-09-13T17:44:09.092808+0000","last_peered":"2024-09-13T17:44:09.092808+0000","last_clean":"2024-09-13T17:44:09.092808+0000","last_became_active":"2024-09-13T17:44:09.092341+0000","last_became_peered":"2024-09-13T17:44:09.092341+0000","last_unstale":"2024-09-13T17:44:09.092808+0000","last_undegraded":"2024-09-13T17:44:09.092808+0000","last_fullsized":"2024-09-13T17:44:09.092808+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:26:57.016183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.089407+0000","last_change":"2024-09-13T17:44:08.083995+0000","last_active":"2024-09-13T17:44:08.089407+0000","last_peered":"2024-09-13T17:44:08.089407+0000","last_clean":"2024-09-13T17:44:08.089407+0000","last_became_active":"2024-09-13T17:44:08.083754+0000","last_became_peered":"2024-09-13T17:44:08.083754+0000","last_unstale":"2024-09-13T17:44:08.089407+0000","last_undegraded":"2024-09-13T17:44:08.089407+0000","last_fullsized":"2024-09-13T17:44:08.089407+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:10.712527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.105073+0000","last_change":"2024-09-13T17:44:09.105073+0000","last_active":"2024-09-13T17:44:09.105073+0000","last_peered":"2024-09-13T17:44:09.105073+0000","last_clean":"2024-09-13T17:44:09.105073+0000","last_became_active":"2024-09-13T17:44:09.104883+0000","last_became_peered":"2024-09-13T17:44:09.104883+0000","last_unstale":"2024-09-13T17:44:09.105073+0000","last_undegraded":"2024-09-13T17:44:09.105073+0000","last_fullsized":"2024-09-13T17:44:09.105073+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:43.983384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.083050+0000","last_change":"2024-09-13T17:44:08.081802+0000","last_active":"2024-09-13T17:44:08.083050+0000","last_peered":"2024-09-13T17:44:08.083050+0000","last_clean":"2024-09-13T17:44:08.083050+0000","last_became_active":"2024-09-13T17:44:08.081561+0000","last_became_peered":"2024-09-13T17:44:08.081561+0000","last_unstale":"2024-09-13T17:44:08.083050+0000","last_undegraded":"2024-09-13T17:44:08.083050+0000","last_fullsized":"2024-09-13T17:44:08.083050+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:00:06.716152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.100180+0000","last_change":"2024-09-13T17:44:09.100180+0000","last_active":"2024-09-13T17:44:09.100180+0000","last_peered":"2024-09-13T17:44:09.100180+0000","last_clean":"2024-09-13T17:44:09.100180+0000","last_became_active":"2024-09-13T17:44:09.097650+0000","last_became_peered":"2024-09-13T17:44:09.097650+0000","last_unstale":"2024-09-13T17:44:09.100180+0000","last_undegraded":"2024-09-13T17:44:09.100180+0000","last_fullsized":"2024-09-13T17:44:09.100180+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:08:00.966936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.082752+0000","last_change":"2024-09-13T17:44:08.080765+0000","last_active":"2024-09-13T17:44:08.082752+0000","last_peered":"2024-09-13T17:44:08.082752+0000","last_clean":"2024-09-13T17:44:08.082752+0000","last_became_active":"2024-09-13T17:44:08.080523+0000","last_became_peered":"2024-09-13T17:44:08.080523+0000","last_unstale":"2024-09-13T17:44:08.082752+0000","last_undegraded":"2024-09-13T17:44:08.082752+0000","last_fullsized":"2024-09-13T17:44:08.082752+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:10:24.427351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.077080+0000","last_change":"2024-09-13T17:44:09.077080+0000","last_active":"2024-09-13T17:44:09.077080+0000","last_peered":"2024-09-13T17:44:09.077080+0000","last_clean":"2024-09-13T17:44:09.077080+0000","last_became_active":"2024-09-13T17:44:09.076529+0000","last_became_peered":"2024-09-13T17:44:09.076529+0000","last_unstale":"2024-09-13T17:44:09.077080+0000","last_undegraded":"2024-09-13T17:44:09.077080+0000","last_fullsized":"2024-09-13T17:44:09.077080+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:38.841320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"31'3","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.099802+0000","last_change":"2024-09-13T17:44:08.080977+0000","last_active":"2024-09-13T17:44:09.099802+0000","last_peered":"2024-09-13T17:44:09.099802+0000","last_clean":"2024-09-13T17:44:09.099802+0000","last_became_active":"2024-09-13T17:44:08.080717+0000","last_became_peered":"2024-09-13T17:44:08.080717+0000","last_unstale":"2024-09-13T17:44:09.099802+0000","last_undegraded":"2024-09-13T17:44:09.099802+0000","last_fullsized":"2024-09-13T17:44:09.099802+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:40:05.058901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.085550+0000","last_change":"2024-09-13T17:44:09.085550+0000","last_active":"2024-09-13T17:44:09.085550+0000","last_peered":"2024-09-13T17:44:09.085550+0000","last_clean":"2024-09-13T17:44:09.085550+0000","last_became_active":"2024-09-13T17:44:09.085317+0000","last_became_peered":"2024-09-13T17:44:09.085317+0000","last_unstale":"2024-09-13T17:44:09.085550+0000","last_undegraded":"2024-09-13T17:44:09.085550+0000","last_fullsized":"2024-09-13T17:44:09.085550+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:10:25.431639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"31'6","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.106835+0000","last_change":"2024-09-13T17:44:08.082800+0000","last_active":"2024-09-13T17:44:09.106835+0000","last_peered":"2024-09-13T17:44:09.106835+0000","last_clean":"2024-09-13T17:44:09.106835+0000","last_became_active":"2024-09-13T17:44:08.082558+0000","last_became_peered":"2024-09-13T17:44:08.082558+0000","last_unstale":"2024-09-13T17:44:09.106835+0000","last_undegraded":"2024-09-13T17:44:09.106835+0000","last_fullsized":"2024-09-13T17:44:09.106835+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:45.793256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.098333+0000","last_change":"2024-09-13T17:44:09.098333+0000","last_active":"2024-09-13T17:44:09.098333+0000","last_peered":"2024-09-13T17:44:09.098333+0000","last_clean":"2024-09-13T17:44:09.098333+0000","last_became_active":"2024-09-13T17:44:09.098092+0000","last_became_peered":"2024-09-13T17:44:09.098092+0000","last_unstale":"2024-09-13T17:44:09.098333+0000","last_undegraded":"2024-09-13T17:44:09.098333+0000","last_fullsized":"2024-09-13T17:44:09.098333+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:17:45.470130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.095273+0000","last_change":"2024-09-13T17:44:08.086760+0000","last_active":"2024-09-13T17:44:09.095273+0000","last_peered":"2024-09-13T17:44:09.095273+0000","last_clean":"2024-09-13T17:44:09.095273+0000","last_became_active":"2024-09-13T17:44:08.086516+0000","last_became_peered":"2024-09-13T17:44:08.086516+0000","last_unstale":"2024-09-13T17:44:09.095273+0000","last_undegraded":"2024-09-13T17:44:09.095273+0000","last_fullsized":"2024-09-13T17:44:09.095273+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:33:03.626924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.095841+0000","last_change":"2024-09-13T17:44:09.095841+0000","last_active":"2024-09-13T17:44:09.095841+0000","last_peered":"2024-09-13T17:44:09.095841+0000","last_clean":"2024-09-13T17:44:09.095841+0000","last_became_active":"2024-09-13T17:44:09.095603+0000","last_became_peered":"2024-09-13T17:44:09.095603+0000","last_unstale":"2024-09-13T17:44:09.095841+0000","last_undegraded":"2024-09-13T17:44:09.095841+0000","last_fullsized":"2024-09-13T17:44:09.095841+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:09:59.370569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.089292+0000","last_change":"2024-09-13T17:44:08.080876+0000","last_active":"2024-09-13T17:44:08.089292+0000","last_peered":"2024-09-13T17:44:08.089292+0000","last_clean":"2024-09-13T17:44:08.089292+0000","last_became_active":"2024-09-13T17:44:08.080073+0000","last_became_peered":"2024-09-13T17:44:08.080073+0000","last_unstale":"2024-09-13T17:44:08.089292+0000","last_undegraded":"2024-09-13T17:44:08.089292+0000","last_fullsized":"2024-09-13T17:44:08.089292+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:00:18.297348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.086890+0000","last_change":"2024-09-13T17:44:09.086890+0000","last_active":"2024-09-13T17:44:09.086890+0000","last_peered":"2024-09-13T17:44:09.086890+0000","last_clean":"2024-09-13T17:44:09.086890+0000","last_became_active":"2024-09-13T17:44:09.086512+0000","last_became_peered":"2024-09-13T17:44:09.086512+0000","last_unstale":"2024-09-13T17:44:09.086890+0000","last_undegraded":"2024-09-13T17:44:09.086890+0000","last_fullsized":"2024-09-13T17:44:09.086890+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:33:04.631213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.086152+0000","last_change":"2024-09-13T17:44:08.082647+0000","last_active":"2024-09-13T17:44:08.086152+0000","last_peered":"2024-09-13T17:44:08.086152+0000","last_clean":"2024-09-13T17:44:08.086152+0000","last_became_active":"2024-09-13T17:44:08.082414+0000","last_became_peered":"2024-09-13T17:44:08.082414+0000","last_unstale":"2024-09-13T17:44:08.086152+0000","last_undegraded":"2024-09-13T17:44:08.086152+0000","last_fullsized":"2024-09-13T17:44:08.086152+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:23:32.691291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.091976+0000","last_change":"2024-09-13T17:44:09.091976+0000","last_active":"2024-09-13T17:44:09.091976+0000","last_peered":"2024-09-13T17:44:09.091976+0000","last_clean":"2024-09-13T17:44:09.091976+0000","last_became_active":"2024-09-13T17:44:09.091714+0000","last_became_peered":"2024-09-13T17:44:09.091714+0000","last_unstale":"2024-09-13T17:44:09.091976+0000","last_undegraded":"2024-09-13T17:44:09.091976+0000","last_fullsized":"2024-09-13T17:44:09.091976+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:14.755193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"31'8","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.105106+0000","last_change":"2024-09-13T17:44:08.080734+0000","last_active":"2024-09-13T17:44:09.105106+0000","last_peered":"2024-09-13T17:44:09.105106+0000","last_clean":"2024-09-13T17:44:09.105106+0000","last_became_active":"2024-09-13T17:44:08.080497+0000","last_became_peered":"2024-09-13T17:44:08.080497+0000","last_unstale":"2024-09-13T17:44:09.105106+0000","last_undegraded":"2024-09-13T17:44:09.105106+0000","last_fullsized":"2024-09-13T17:44:09.105106+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:11:10.137498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.092611+0000","last_change":"2024-09-13T17:44:09.092611+0000","last_active":"2024-09-13T17:44:09.092611+0000","last_peered":"2024-09-13T17:44:09.092611+0000","last_clean":"2024-09-13T17:44:09.092611+0000","last_became_active":"2024-09-13T17:44:09.091079+0000","last_became_peered":"2024-09-13T17:44:09.091079+0000","last_unstale":"2024-09-13T17:44:09.092611+0000","last_undegraded":"2024-09-13T17:44:09.092611+0000","last_fullsized":"2024-09-13T17:44:09.092611+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:53:57.379081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"31'1","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.063609+0000","last_change":"2024-09-13T17:44:08.084294+0000","last_active":"2024-09-13T17:44:09.063609+0000","last_peered":"2024-09-13T17:44:09.063609+0000","last_clean":"2024-09-13T17:44:09.063609+0000","last_became_active":"2024-09-13T17:44:08.084033+0000","last_became_peered":"2024-09-13T17:44:08.084033+0000","last_unstale":"2024-09-13T17:44:09.063609+0000","last_undegraded":"2024-09-13T17:44:09.063609+0000","last_fullsized":"2024-09-13T17:44:09.063609+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:04:26.313895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.081895+0000","last_change":"2024-09-13T17:44:09.081895+0000","last_active":"2024-09-13T17:44:09.081895+0000","last_peered":"2024-09-13T17:44:09.081895+0000","last_clean":"2024-09-13T17:44:09.081895+0000","last_became_active":"2024-09-13T17:44:09.080183+0000","last_became_peered":"2024-09-13T17:44:09.080183+0000","last_unstale":"2024-09-13T17:44:09.081895+0000","last_undegraded":"2024-09-13T17:44:09.081895+0000","last_fullsized":"2024-09-13T17:44:09.081895+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:33:04.631213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.102962+0000","last_change":"2024-09-13T17:44:08.091269+0000","last_active":"2024-09-13T17:44:09.102962+0000","last_peered":"2024-09-13T17:44:09.102962+0000","last_clean":"2024-09-13T17:44:09.102962+0000","last_became_active":"2024-09-13T17:44:08.091000+0000","last_became_peered":"2024-09-13T17:44:08.091000+0000","last_unstale":"2024-09-13T17:44:09.102962+0000","last_undegraded":"2024-09-13T17:44:09.102962+0000","last_fullsized":"2024-09-13T17:44:09.102962+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:47.848588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.085165+0000","last_change":"2024-09-13T17:44:09.085165+0000","last_active":"2024-09-13T17:44:09.085165+0000","last_peered":"2024-09-13T17:44:09.085165+0000","last_clean":"2024-09-13T17:44:09.085165+0000","last_became_active":"2024-09-13T17:44:09.084779+0000","last_became_peered":"2024-09-13T17:44:09.084779+0000","last_unstale":"2024-09-13T17:44:09.085165+0000","last_undegraded":"2024-09-13T17:44:09.085165+0000","last_fullsized":"2024-09-13T17:44:09.085165+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:23:36.075940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"31'1","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.073400+0000","last_change":"2024-09-13T17:44:08.086166+0000","last_active":"2024-09-13T17:44:09.073400+0000","last_peered":"2024-09-13T17:44:09.073400+0000","last_clean":"2024-09-13T17:44:09.073400+0000","last_became_active":"2024-09-13T17:44:08.085913+0000","last_became_peered":"2024-09-13T17:44:08.085913+0000","last_unstale":"2024-09-13T17:44:09.073400+0000","last_undegraded":"2024-09-13T17:44:09.073400+0000","last_fullsized":"2024-09-13T17:44:09.073400+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:17:08.989245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.101360+0000","last_change":"2024-09-13T17:44:09.101360+0000","last_active":"2024-09-13T17:44:09.101360+0000","last_peered":"2024-09-13T17:44:09.101360+0000","last_clean":"2024-09-13T17:44:09.101360+0000","last_became_active":"2024-09-13T17:44:09.101119+0000","last_became_peered":"2024-09-13T17:44:09.101119+0000","last_unstale":"2024-09-13T17:44:09.101360+0000","last_undegraded":"2024-09-13T17:44:09.101360+0000","last_fullsized":"2024-09-13T17:44:09.101360+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:18:08.511918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.092056+0000","last_change":"2024-09-13T17:44:08.084464+0000","last_active":"2024-09-13T17:44:08.092056+0000","last_peered":"2024-09-13T17:44:08.092056+0000","last_clean":"2024-09-13T17:44:08.092056+0000","last_became_active":"2024-09-13T17:44:08.084233+0000","last_became_peered":"2024-09-13T17:44:08.084233+0000","last_unstale":"2024-09-13T17:44:08.092056+0000","last_undegraded":"2024-09-13T17:44:08.092056+0000","last_fullsized":"2024-09-13T17:44:08.092056+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:15.312171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.091582+0000","last_change":"2024-09-13T17:44:09.091582+0000","last_active":"2024-09-13T17:44:09.091582+0000","last_peered":"2024-09-13T17:44:09.091582+0000","last_clean":"2024-09-13T17:44:09.091582+0000","last_became_active":"2024-09-13T17:44:09.091358+0000","last_became_peered":"2024-09-13T17:44:09.091358+0000","last_unstale":"2024-09-13T17:44:09.091582+0000","last_undegraded":"2024-09-13T17:44:09.091582+0000","last_fullsized":"2024-09-13T17:44:09.091582+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:51:47.981712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.104879+0000","last_change":"2024-09-13T17:44:08.087156+0000","last_active":"2024-09-13T17:44:09.104879+0000","last_peered":"2024-09-13T17:44:09.104879+0000","last_clean":"2024-09-13T17:44:09.104879+0000","last_became_active":"2024-09-13T17:44:08.086617+0000","last_became_peered":"2024-09-13T17:44:08.086617+0000","last_unstale":"2024-09-13T17:44:09.104879+0000","last_undegraded":"2024-09-13T17:44:09.104879+0000","last_fullsized":"2024-09-13T17:44:09.104879+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:01:51.213503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.078569+0000","last_change":"2024-09-13T17:44:09.078569+0000","last_active":"2024-09-13T17:44:09.078569+0000","last_peered":"2024-09-13T17:44:09.078569+0000","last_clean":"2024-09-13T17:44:09.078569+0000","last_became_active":"2024-09-13T17:44:09.078028+0000","last_became_peered":"2024-09-13T17:44:09.078028+0000","last_unstale":"2024-09-13T17:44:09.078569+0000","last_undegraded":"2024-09-13T17:44:09.078569+0000","last_fullsized":"2024-09-13T17:44:09.078569+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:00.260017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.092168+0000","last_change":"2024-09-13T17:44:08.090522+0000","last_active":"2024-09-13T17:44:08.092168+0000","last_peered":"2024-09-13T17:44:08.092168+0000","last_clean":"2024-09-13T17:44:08.092168+0000","last_became_active":"2024-09-13T17:44:08.090231+0000","last_became_peered":"2024-09-13T17:44:08.090231+0000","last_unstale":"2024-09-13T17:44:08.092168+0000","last_undegraded":"2024-09-13T17:44:08.092168+0000","last_fullsized":"2024-09-13T17:44:08.092168+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:48.658048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.076366+0000","last_change":"2024-09-13T17:44:09.076366+0000","last_active":"2024-09-13T17:44:09.076366+0000","last_peered":"2024-09-13T17:44:09.076366+0000","last_clean":"2024-09-13T17:44:09.076366+0000","last_became_active":"2024-09-13T17:44:09.075083+0000","last_became_peered":"2024-09-13T17:44:09.075083+0000","last_unstale":"2024-09-13T17:44:09.076366+0000","last_undegraded":"2024-09-13T17:44:09.076366+0000","last_fullsized":"2024-09-13T17:44:09.076366+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:18:08.511918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.089264+0000","last_change":"2024-09-13T17:44:08.084135+0000","last_active":"2024-09-13T17:44:08.089264+0000","last_peered":"2024-09-13T17:44:08.089264+0000","last_clean":"2024-09-13T17:44:08.089264+0000","last_became_active":"2024-09-13T17:44:08.083885+0000","last_became_peered":"2024-09-13T17:44:08.083885+0000","last_unstale":"2024-09-13T17:44:08.089264+0000","last_undegraded":"2024-09-13T17:44:08.089264+0000","last_fullsized":"2024-09-13T17:44:08.089264+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:55.216920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.084497+0000","last_change":"2024-09-13T17:44:09.084497+0000","last_active":"2024-09-13T17:44:09.084497+0000","last_peered":"2024-09-13T17:44:09.084497+0000","last_clean":"2024-09-13T17:44:09.084497+0000","last_became_active":"2024-09-13T17:44:09.081961+0000","last_became_peered":"2024-09-13T17:44:09.081961+0000","last_unstale":"2024-09-13T17:44:09.084497+0000","last_undegraded":"2024-09-13T17:44:09.084497+0000","last_fullsized":"2024-09-13T17:44:09.084497+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:06:24.257175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.087438+0000","last_change":"2024-09-13T17:44:08.085352+0000","last_active":"2024-09-13T17:44:08.087438+0000","last_peered":"2024-09-13T17:44:08.087438+0000","last_clean":"2024-09-13T17:44:08.087438+0000","last_became_active":"2024-09-13T17:44:08.085113+0000","last_became_peered":"2024-09-13T17:44:08.085113+0000","last_unstale":"2024-09-13T17:44:08.087438+0000","last_undegraded":"2024-09-13T17:44:08.087438+0000","last_fullsized":"2024-09-13T17:44:08.087438+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:29:49.837938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.092521+0000","last_change":"2024-09-13T17:44:09.092521+0000","last_active":"2024-09-13T17:44:09.092521+0000","last_peered":"2024-09-13T17:44:09.092521+0000","last_clean":"2024-09-13T17:44:09.092521+0000","last_became_active":"2024-09-13T17:44:09.092212+0000","last_became_peered":"2024-09-13T17:44:09.092212+0000","last_unstale":"2024-09-13T17:44:09.092521+0000","last_undegraded":"2024-09-13T17:44:09.092521+0000","last_fullsized":"2024-09-13T17:44:09.092521+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:52.647827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.088970+0000","last_change":"2024-09-13T17:44:08.084591+0000","last_active":"2024-09-13T17:44:08.088970+0000","last_peered":"2024-09-13T17:44:08.088970+0000","last_clean":"2024-09-13T17:44:08.088970+0000","last_became_active":"2024-09-13T17:44:08.084345+0000","last_became_peered":"2024-09-13T17:44:08.084345+0000","last_unstale":"2024-09-13T17:44:08.088970+0000","last_undegraded":"2024-09-13T17:44:08.088970+0000","last_fullsized":"2024-09-13T17:44:08.088970+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:44:30.090059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":94,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.071200+0000","last_change":"2024-09-13T17:41:55.654539+0000","last_active":"2024-09-13T17:44:08.071200+0000","last_peered":"2024-09-13T17:44:08.071200+0000","last_clean":"2024-09-13T17:44:08.071200+0000","last_became_active":"2024-09-13T17:41:55.653536+0000","last_became_peered":"2024-09-13T17:41:55.653536+0000","last_unstale":"2024-09-13T17:44:08.071200+0000","last_undegraded":"2024-09-13T17:44:08.071200+0000","last_fullsized":"2024-09-13T17:44:08.071200+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:54.532965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:54.532965+0000","last_clean_scrub_stamp":"2024-09-13T17:41:54.532965+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:56.662588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":30,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.069418+0000","last_change":"2024-09-13T17:42:02.137356+0000","last_active":"2024-09-13T17:44:08.069418+0000","last_peered":"2024-09-13T17:44:08.069418+0000","last_clean":"2024-09-13T17:44:08.069418+0000","last_became_active":"2024-09-13T17:42:00.116534+0000","last_became_peered":"2024-09-13T17:42:00.116534+0000","last_unstale":"2024-09-13T17:44:08.069418+0000","last_undegraded":"2024-09-13T17:44:08.069418+0000","last_fullsized":"2024-09-13T17:44:08.069418+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:06.665067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000321799,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.076914+0000","last_change":"2024-09-13T17:44:09.076914+0000","last_active":"2024-09-13T17:44:09.076914+0000","last_peered":"2024-09-13T17:44:09.076914+0000","last_clean":"2024-09-13T17:44:09.076914+0000","last_became_active":"2024-09-13T17:44:09.076704+0000","last_became_peered":"2024-09-13T17:44:09.076704+0000","last_unstale":"2024-09-13T17:44:09.076914+0000","last_undegraded":"2024-09-13T17:44:09.076914+0000","last_fullsized":"2024-09-13T17:44:09.076914+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:00.260017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.099117+0000","last_change":"2024-09-13T17:44:08.081723+0000","last_active":"2024-09-13T17:44:09.099117+0000","last_peered":"2024-09-13T17:44:09.099117+0000","last_clean":"2024-09-13T17:44:09.099117+0000","last_became_active":"2024-09-13T17:44:08.081265+0000","last_became_peered":"2024-09-13T17:44:08.081265+0000","last_unstale":"2024-09-13T17:44:09.099117+0000","last_undegraded":"2024-09-13T17:44:09.099117+0000","last_fullsized":"2024-09-13T17:44:09.099117+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:28.105601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":29,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.067852+0000","last_change":"2024-09-13T17:42:02.137373+0000","last_active":"2024-09-13T17:44:08.067852+0000","last_peered":"2024-09-13T17:44:08.067852+0000","last_clean":"2024-09-13T17:44:08.067852+0000","last_became_active":"2024-09-13T17:42:00.116542+0000","last_became_peered":"2024-09-13T17:42:00.116542+0000","last_unstale":"2024-09-13T17:44:08.067852+0000","last_undegraded":"2024-09-13T17:44:08.067852+0000","last_fullsized":"2024-09-13T17:44:08.067852+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:53.754563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035063,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.080414+0000","last_change":"2024-09-13T17:44:09.080414+0000","last_active":"2024-09-13T17:44:09.080414+0000","last_peered":"2024-09-13T17:44:09.080414+0000","last_clean":"2024-09-13T17:44:09.080414+0000","last_became_active":"2024-09-13T17:44:09.080067+0000","last_became_peered":"2024-09-13T17:44:09.080067+0000","last_unstale":"2024-09-13T17:44:09.080414+0000","last_undegraded":"2024-09-13T17:44:09.080414+0000","last_fullsized":"2024-09-13T17:44:09.080414+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:46.797545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.091733+0000","last_change":"2024-09-13T17:44:08.089279+0000","last_active":"2024-09-13T17:44:08.091733+0000","last_peered":"2024-09-13T17:44:08.091733+0000","last_clean":"2024-09-13T17:44:08.091733+0000","last_became_active":"2024-09-13T17:44:08.089033+0000","last_became_peered":"2024-09-13T17:44:08.089033+0000","last_unstale":"2024-09-13T17:44:08.091733+0000","last_undegraded":"2024-09-13T17:44:08.091733+0000","last_fullsized":"2024-09-13T17:44:08.091733+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:48:54.288787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.066524+0000","last_change":"2024-09-13T17:42:02.615831+0000","last_active":"2024-09-13T17:44:08.066524+0000","last_peered":"2024-09-13T17:44:08.066524+0000","last_clean":"2024-09-13T17:44:08.066524+0000","last_became_active":"2024-09-13T17:42:00.116427+0000","last_became_peered":"2024-09-13T17:42:00.116427+0000","last_unstale":"2024-09-13T17:44:08.066524+0000","last_undegraded":"2024-09-13T17:44:08.066524+0000","last_fullsized":"2024-09-13T17:44:08.066524+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:29:17.848348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000228423,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.095690+0000","last_change":"2024-09-13T17:44:09.095690+0000","last_active":"2024-09-13T17:44:09.095690+0000","last_peered":"2024-09-13T17:44:09.095690+0000","last_clean":"2024-09-13T17:44:09.095690+0000","last_became_active":"2024-09-13T17:44:09.095448+0000","last_became_peered":"2024-09-13T17:44:09.095448+0000","last_unstale":"2024-09-13T17:44:09.095690+0000","last_undegraded":"2024-09-13T17:44:09.095690+0000","last_fullsized":"2024-09-13T17:44:09.095690+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:12:54.554173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.084187+0000","last_change":"2024-09-13T17:44:08.080099+0000","last_active":"2024-09-13T17:44:08.084187+0000","last_peered":"2024-09-13T17:44:08.084187+0000","last_clean":"2024-09-13T17:44:08.084187+0000","last_became_active":"2024-09-13T17:44:08.079856+0000","last_became_peered":"2024-09-13T17:44:08.079856+0000","last_unstale":"2024-09-13T17:44:08.084187+0000","last_undegraded":"2024-09-13T17:44:08.084187+0000","last_fullsized":"2024-09-13T17:44:08.084187+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:59.140826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":31,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.067921+0000","last_change":"2024-09-13T17:42:02.138081+0000","last_active":"2024-09-13T17:44:08.067921+0000","last_peered":"2024-09-13T17:44:08.067921+0000","last_clean":"2024-09-13T17:44:08.067921+0000","last_became_active":"2024-09-13T17:42:00.116465+0000","last_became_peered":"2024-09-13T17:42:00.116465+0000","last_unstale":"2024-09-13T17:44:08.067921+0000","last_undegraded":"2024-09-13T17:44:08.067921+0000","last_fullsized":"2024-09-13T17:44:08.067921+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:40:30.342531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001026573,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.085889+0000","last_change":"2024-09-13T17:44:09.085889+0000","last_active":"2024-09-13T17:44:09.085889+0000","last_peered":"2024-09-13T17:44:09.085889+0000","last_clean":"2024-09-13T17:44:09.085889+0000","last_became_active":"2024-09-13T17:44:09.085652+0000","last_became_peered":"2024-09-13T17:44:09.085652+0000","last_unstale":"2024-09-13T17:44:09.085889+0000","last_undegraded":"2024-09-13T17:44:09.085889+0000","last_fullsized":"2024-09-13T17:44:09.085889+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:44.944657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.091513+0000","last_change":"2024-09-13T17:44:08.086787+0000","last_active":"2024-09-13T17:44:08.091513+0000","last_peered":"2024-09-13T17:44:08.091513+0000","last_clean":"2024-09-13T17:44:08.091513+0000","last_became_active":"2024-09-13T17:44:08.086546+0000","last_became_peered":"2024-09-13T17:44:08.086546+0000","last_unstale":"2024-09-13T17:44:08.091513+0000","last_undegraded":"2024-09-13T17:44:08.091513+0000","last_fullsized":"2024-09-13T17:44:08.091513+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:31.939247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.073003+0000","last_change":"2024-09-13T17:42:02.135135+0000","last_active":"2024-09-13T17:44:08.073003+0000","last_peered":"2024-09-13T17:44:08.073003+0000","last_clean":"2024-09-13T17:44:08.073003+0000","last_became_active":"2024-09-13T17:42:00.364240+0000","last_became_peered":"2024-09-13T17:42:00.364240+0000","last_unstale":"2024-09-13T17:44:08.073003+0000","last_undegraded":"2024-09-13T17:44:08.073003+0000","last_fullsized":"2024-09-13T17:44:08.073003+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:31:29.382006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575944,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.068199+0000","last_change":"2024-09-13T17:44:09.068199+0000","last_active":"2024-09-13T17:44:09.068199+0000","last_peered":"2024-09-13T17:44:09.068199+0000","last_clean":"2024-09-13T17:44:09.068199+0000","last_became_active":"2024-09-13T17:44:09.067939+0000","last_became_peered":"2024-09-13T17:44:09.067939+0000","last_unstale":"2024-09-13T17:44:09.068199+0000","last_undegraded":"2024-09-13T17:44:09.068199+0000","last_fullsized":"2024-09-13T17:44:09.068199+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:08:00.966936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.091636+0000","last_change":"2024-09-13T17:44:08.088268+0000","last_active":"2024-09-13T17:44:08.091636+0000","last_peered":"2024-09-13T17:44:08.091636+0000","last_clean":"2024-09-13T17:44:08.091636+0000","last_became_active":"2024-09-13T17:44:08.087998+0000","last_became_peered":"2024-09-13T17:44:08.087998+0000","last_unstale":"2024-09-13T17:44:08.091636+0000","last_undegraded":"2024-09-13T17:44:08.091636+0000","last_fullsized":"2024-09-13T17:44:08.091636+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:55.216920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":29,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.071260+0000","last_change":"2024-09-13T17:42:02.138136+0000","last_active":"2024-09-13T17:44:08.071260+0000","last_peered":"2024-09-13T17:44:08.071260+0000","last_clean":"2024-09-13T17:44:08.071260+0000","last_became_active":"2024-09-13T17:42:00.364233+0000","last_became_peered":"2024-09-13T17:42:00.364233+0000","last_unstale":"2024-09-13T17:44:08.071260+0000","last_undegraded":"2024-09-13T17:44:08.071260+0000","last_fullsized":"2024-09-13T17:44:08.071260+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:54.234194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010757130000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.092485+0000","last_change":"2024-09-13T17:44:09.092485+0000","last_active":"2024-09-13T17:44:09.092485+0000","last_peered":"2024-09-13T17:44:09.092485+0000","last_clean":"2024-09-13T17:44:09.092485+0000","last_became_active":"2024-09-13T17:44:09.092232+0000","last_became_peered":"2024-09-13T17:44:09.092232+0000","last_unstale":"2024-09-13T17:44:09.092485+0000","last_undegraded":"2024-09-13T17:44:09.092485+0000","last_fullsized":"2024-09-13T17:44:09.092485+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:47.365057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.094767+0000","last_change":"2024-09-13T17:44:08.091205+0000","last_active":"2024-09-13T17:44:08.094767+0000","last_peered":"2024-09-13T17:44:08.094767+0000","last_clean":"2024-09-13T17:44:08.094767+0000","last_became_active":"2024-09-13T17:44:08.090931+0000","last_became_peered":"2024-09-13T17:44:08.090931+0000","last_unstale":"2024-09-13T17:44:08.094767+0000","last_undegraded":"2024-09-13T17:44:08.094767+0000","last_fullsized":"2024-09-13T17:44:08.094767+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:43.940369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":29,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.070560+0000","last_change":"2024-09-13T17:42:02.134963+0000","last_active":"2024-09-13T17:44:08.070560+0000","last_peered":"2024-09-13T17:44:08.070560+0000","last_clean":"2024-09-13T17:44:08.070560+0000","last_became_active":"2024-09-13T17:42:00.364233+0000","last_became_peered":"2024-09-13T17:42:00.364233+0000","last_unstale":"2024-09-13T17:44:08.070560+0000","last_undegraded":"2024-09-13T17:44:08.070560+0000","last_fullsized":"2024-09-13T17:44:08.070560+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:24:31.191067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393223,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.095839+0000","last_change":"2024-09-13T17:44:09.095839+0000","last_active":"2024-09-13T17:44:09.095839+0000","last_peered":"2024-09-13T17:44:09.095839+0000","last_clean":"2024-09-13T17:44:09.095839+0000","last_became_active":"2024-09-13T17:44:09.095323+0000","last_became_peered":"2024-09-13T17:44:09.095323+0000","last_unstale":"2024-09-13T17:44:09.095839+0000","last_undegraded":"2024-09-13T17:44:09.095839+0000","last_fullsized":"2024-09-13T17:44:09.095839+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:23:36.075940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.082770+0000","last_change":"2024-09-13T17:44:08.080518+0000","last_active":"2024-09-13T17:44:08.082770+0000","last_peered":"2024-09-13T17:44:08.082770+0000","last_clean":"2024-09-13T17:44:08.082770+0000","last_became_active":"2024-09-13T17:44:08.080255+0000","last_became_peered":"2024-09-13T17:44:08.080255+0000","last_unstale":"2024-09-13T17:44:08.082770+0000","last_undegraded":"2024-09-13T17:44:08.082770+0000","last_fullsized":"2024-09-13T17:44:08.082770+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:23:35.071651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":29,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.071822+0000","last_change":"2024-09-13T17:42:02.138362+0000","last_active":"2024-09-13T17:44:08.071822+0000","last_peered":"2024-09-13T17:44:08.071822+0000","last_clean":"2024-09-13T17:44:08.071822+0000","last_became_active":"2024-09-13T17:42:00.364214+0000","last_became_peered":"2024-09-13T17:42:00.364214+0000","last_unstale":"2024-09-13T17:44:08.071822+0000","last_undegraded":"2024-09-13T17:44:08.071822+0000","last_fullsized":"2024-09-13T17:44:08.071822+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:33.976642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012776910000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705694,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297620,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074220,"statfs":{"total":96636764160,"available":96332001280,"internally_reserved":0,"allocated":266240,"data_stored":93848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:44:00 2024","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.30399999999999999,"15min":0.30499999999999999},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.39800000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.29299999999999998,"5min":0.317,"15min":0.32100000000000001},"min":{"1min":0.223,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.45300000000000001,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.35399999999999998}]},{"osd":1,"last update":"Fri Sep 13 17:44:00 2024","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.34899999999999998,"15min":0.35299999999999998},"min":{"1min":0.255,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.5,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.30599999999999999,"5min":0.318,"15min":0.32000000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.55200000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.33800000000000002}]},{"osd":2,"last update":"Fri Sep 13 17:44:00 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.36899999999999999,"15min":0.372},"min":{"1min":0.28000000000000003,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.51100000000000001,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.33700000000000002,"15min":0.33800000000000002},"min":{"1min":0.23499999999999999,"5min":0.216,"15min":0.216},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.40699999999999997}]}]},{"osd":2,"up_from":9,"seq":38654705694,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297616,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074224,"statfs":{"total":96636764160,"available":96332005376,"internally_reserved":0,"allocated":262144,"data_stored":93829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:43:59 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.31,"15min":0.307},"min":{"1min":0.22500000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.34599999999999997,"15min":0.34999999999999998},"min":{"1min":0.22700000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.30099999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:43:59 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35199999999999998,"15min":0.35299999999999998},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.252},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.35699999999999998,"15min":0.35799999999999998},"min":{"1min":0.24399999999999999,"5min":0.224,"15min":0.224},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.32100000000000001}]},{"osd":3,"last update":"Fri Sep 13 17:43:59 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.377,"15min":0.379},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.376,"5min":0.372,"15min":0.371},"min":{"1min":0.28599999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.28699999999999998}]}]},{"osd":1,"up_from":9,"seq":38654705697,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298216,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073624,"statfs":{"total":96636764160,"available":96331390976,"internally_reserved":0,"allocated":880640,"data_stored":689584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:43:56 2024","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.40300000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.31,"15min":0.307},"min":{"1min":0.25,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"last":0.27700000000000002}]},{"osd":2,"last update":"Fri Sep 13 17:43:56 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.318,"15min":0.317},"min":{"1min":0.23200000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"last":0.248},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.251,"5min":0.222,"15min":0.222},"max":{"1min":0.44800000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"last":0.30299999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:43:56 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.34799999999999998,"15min":0.34999999999999998},"min":{"1min":0.246,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"last":0.312},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35999999999999999,"15min":0.36099999999999999},"min":{"1min":0.26800000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.46500000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.29499999999999998}]}]},{"osd":0,"up_from":9,"seq":38654705695,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298196,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073644,"statfs":{"total":96636764160,"available":96331411456,"internally_reserved":0,"allocated":856064,"data_stored":684197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:43:57 2024","interfaces":[{"interface":"back","average":{"1min":0.26700000000000002,"5min":0.29999999999999999,"15min":0.30599999999999999},"min":{"1min":0.214,"5min":0.19,"15min":0.19},"max":{"1min":0.38200000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.307,"5min":0.311,"15min":0.311},"min":{"1min":0.22900000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.42299999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"last":0.28999999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:43:57 2024","interfaces":[{"interface":"back","average":{"1min":0.29799999999999999,"5min":0.318,"15min":0.32100000000000001},"min":{"1min":0.23499999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.41599999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.31,"5min":0.32400000000000001,"15min":0.32600000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.437,"5min":0.44600000000000001,"15min":0.44600000000000001},"last":0.249}]},{"osd":3,"last update":"Fri Sep 13 17:43:57 2024","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.33200000000000002,"15min":0.33300000000000002},"min":{"1min":0.249,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.39700000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.316,"5min":0.33200000000000002,"15min":0.33500000000000002},"min":{"1min":0.254,"5min":0.248,"15min":0.248},"max":{"1min":0.40999999999999998,"5min":0.504,"15min":0.504},"last":0.23499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:44:11.956 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:44:12.043 INFO:tasks.ceph.mds.a.smithi102.stderr:2024-09-13T17:44:12.040+0000 7f1c34ff1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:44:12.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.083+0000 7f6c68634640 1 Processor -- start 2024-09-13T17:44:12.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.083+0000 7f6c68634640 1 -- start start 2024-09-13T17:44:12.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.083+0000 7f6c68634640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6c60128330 0x7f6c60128700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:12.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.083+0000 7f6c68634640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6c60128c40 0x7f6c6012b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:12.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.083+0000 7f6c68634640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c6012bd80 0x7f6c6012c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:12.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.083+0000 7f6c68634640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c6012c760 con 0x7f6c60128c40 2024-09-13T17:44:12.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.083+0000 7f6c68634640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c60097be0 con 0x7f6c6012bd80 2024-09-13T17:44:12.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.083+0000 7f6c68634640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6c60098e70 con 0x7f6c60128330 2024-09-13T17:44:12.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.084+0000 7f6c663d0640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6c60128330 0x7f6c60128700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:12.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.084+0000 7f6c65bcf640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6c60128c40 0x7f6c6012b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:12.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.084+0000 7f6c66bd1640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c6012bd80 0x7f6c6012c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:12.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.084+0000 7f6c663d0640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6c60128330 0x7f6c60128700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:57660/0 (socket says 172.21.15.102:57660) 2024-09-13T17:44:12.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.084+0000 7f6c663d0640 1 -- 172.21.15.102:0/1705830170 learned_addr learned my addr 172.21.15.102:0/1705830170 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:12.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.084+0000 7f6c65bcf640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6c60128c40 0x7f6c6012b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56882/0 (socket says 172.21.15.102:56882) 2024-09-13T17:44:12.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.084+0000 7f6c66bd1640 1 -- 172.21.15.102:0/1705830170 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6c60128330 msgr2=0x7f6c60128700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:12.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.084+0000 7f6c66bd1640 1 --2- 172.21.15.102:0/1705830170 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6c60128330 0x7f6c60128700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.084+0000 7f6c66bd1640 1 -- 172.21.15.102:0/1705830170 >> v2:172.21.15.102:3300/0 conn(0x7f6c60128c40 msgr2=0x7f6c6012b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:12.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.084+0000 7f6c66bd1640 1 --2- 172.21.15.102:0/1705830170 >> v2:172.21.15.102:3300/0 conn(0x7f6c60128c40 0x7f6c6012b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.084+0000 7f6c66bd1640 1 -- 172.21.15.102:0/1705830170 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c60098310 con 0x7f6c6012bd80 2024-09-13T17:44:12.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.084+0000 7f6c663d0640 1 --2- 172.21.15.102:0/1705830170 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6c60128330 0x7f6c60128700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:12.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.084+0000 7f6c65bcf640 1 --2- 172.21.15.102:0/1705830170 >> v2:172.21.15.102:3300/0 conn(0x7f6c60128c40 0x7f6c6012b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:12.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.085+0000 7f6c66bd1640 1 --2- 172.21.15.102:0/1705830170 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c6012bd80 0x7f6c6012c220 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6c58009e60 tx=0x7f6c58009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97a04a60da678a1d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:12.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.085+0000 7f6c653ce640 1 -- 172.21.15.102:0/1705830170 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5802aea0 con 0x7f6c6012bd80 2024-09-13T17:44:12.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.085+0000 7f6c653ce640 1 -- 172.21.15.102:0/1705830170 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5802d050 con 0x7f6c6012bd80 2024-09-13T17:44:12.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.085+0000 7f6c653ce640 1 -- 172.21.15.102:0/1705830170 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c58007780 con 0x7f6c6012bd80 2024-09-13T17:44:12.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.085+0000 7f6c68634640 1 -- 172.21.15.102:0/1705830170 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c6012bd80 msgr2=0x7f6c6012c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:12.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.085+0000 7f6c68634640 1 --2- 172.21.15.102:0/1705830170 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c6012bd80 0x7f6c6012c220 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6c58009e60 tx=0x7f6c58009780 comp rx=0 tx=0).stop 2024-09-13T17:44:12.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.085+0000 7f6c68634640 1 -- 172.21.15.102:0/1705830170 shutdown_connections 2024-09-13T17:44:12.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.085+0000 7f6c68634640 1 --2- 172.21.15.102:0/1705830170 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c6012bd80 0x7f6c6012c220 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.085+0000 7f6c68634640 1 --2- 172.21.15.102:0/1705830170 >> v2:172.21.15.102:3300/0 conn(0x7f6c60128c40 0x7f6c6012b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.085+0000 7f6c68634640 1 --2- 172.21.15.102:0/1705830170 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6c60128330 0x7f6c60128700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.085+0000 7f6c68634640 1 -- 172.21.15.102:0/1705830170 >> 172.21.15.102:0/1705830170 conn(0x7f6c6009e820 msgr2=0x7f6c6009ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:12.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.085+0000 7f6c68634640 1 -- 172.21.15.102:0/1705830170 shutdown_connections 2024-09-13T17:44:12.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.085+0000 7f6c68634640 1 -- 172.21.15.102:0/1705830170 wait complete. 2024-09-13T17:44:12.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.086+0000 7f6c68634640 1 Processor -- start 2024-09-13T17:44:12.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.086+0000 7f6c68634640 1 -- start start 2024-09-13T17:44:12.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.087+0000 7f6c68634640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6c60128330 0x7f6c60114e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:12.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.087+0000 7f6c68634640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c60128c40 0x7f6c601153c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:12.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.087+0000 7f6c68634640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c6012bd80 0x7f6c60115900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:12.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.087+0000 7f6c68634640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6006e1b0 con 0x7f6c60128c40 2024-09-13T17:44:12.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.087+0000 7f6c68634640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c60117c50 con 0x7f6c6012bd80 2024-09-13T17:44:12.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.087+0000 7f6c68634640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6c60117080 con 0x7f6c60128330 2024-09-13T17:44:12.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.087+0000 7f6c663d0640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6c60128330 0x7f6c60114e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:12.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.087+0000 7f6c663d0640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6c60128330 0x7f6c60114e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:57674/0 (socket says 172.21.15.102:57674) 2024-09-13T17:44:12.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.087+0000 7f6c663d0640 1 -- 172.21.15.102:0/1506776880 learned_addr learned my addr 172.21.15.102:0/1506776880 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:12.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.087+0000 7f6c65bcf640 1 --2- 172.21.15.102:0/1506776880 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c60128c40 0x7f6c601153c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:12.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.087+0000 7f6c66bd1640 1 --2- 172.21.15.102:0/1506776880 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c6012bd80 0x7f6c60115900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:12.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.087+0000 7f6c663d0640 1 -- 172.21.15.102:0/1506776880 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c6012bd80 msgr2=0x7f6c60115900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:12.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.087+0000 7f6c663d0640 1 --2- 172.21.15.102:0/1506776880 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c6012bd80 0x7f6c60115900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.087+0000 7f6c663d0640 1 -- 172.21.15.102:0/1506776880 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c60128c40 msgr2=0x7f6c601153c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:12.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.087+0000 7f6c663d0640 1 --2- 172.21.15.102:0/1506776880 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c60128c40 0x7f6c601153c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.087+0000 7f6c663d0640 1 -- 172.21.15.102:0/1506776880 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c5c00b6f0 con 0x7f6c60128330 2024-09-13T17:44:12.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.087+0000 7f6c66bd1640 1 --2- 172.21.15.102:0/1506776880 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c6012bd80 0x7f6c60115900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:12.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.087+0000 7f6c65bcf640 1 --2- 172.21.15.102:0/1506776880 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c60128c40 0x7f6c601153c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:12.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.087+0000 7f6c663d0640 1 --2- 172.21.15.102:0/1506776880 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6c60128330 0x7f6c60114e80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c0100f0 tx=0x7f6c5c00f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:12.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.087+0000 7f6c567fc640 1 -- 172.21.15.102:0/1506776880 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c012040 con 0x7f6c60128330 2024-09-13T17:44:12.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.087+0000 7f6c567fc640 1 -- 172.21.15.102:0/1506776880 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5c010ed0 con 0x7f6c60128330 2024-09-13T17:44:12.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.087+0000 7f6c68634640 1 -- 172.21.15.102:0/1506776880 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c50009390 con 0x7f6c60128330 2024-09-13T17:44:12.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.087+0000 7f6c567fc640 1 -- 172.21.15.102:0/1506776880 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c018620 con 0x7f6c60128330 2024-09-13T17:44:12.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.087+0000 7f6c68634640 1 -- 172.21.15.102:0/1506776880 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c58009a20 con 0x7f6c60128330 2024-09-13T17:44:12.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.088+0000 7f6c567fc640 1 -- 172.21.15.102:0/1506776880 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f6c5c018a50 con 0x7f6c60128330 2024-09-13T17:44:12.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.088+0000 7f6c567fc640 1 -- 172.21.15.102:0/1506776880 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3660+0+0 (secure 0 0 0) 0x7f6c5c044550 con 0x7f6c60128330 2024-09-13T17:44:12.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.088+0000 7f6c68634640 1 -- 172.21.15.102:0/1506776880 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c60064000 con 0x7f6c60128330 2024-09-13T17:44:12.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.094+0000 7f6c567fc640 1 -- 172.21.15.102:0/1506776880 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f6c5c021c80 con 0x7f6c60128330 2024-09-13T17:44:12.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.242+0000 7f6c68634640 1 -- 172.21.15.102:0/1506776880 --> v1:172.21.15.102:6816/34937 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6c601173d0 con 0x7f6c44031160 2024-09-13T17:44:12.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.248+0000 7f6c567fc640 1 -- 172.21.15.102:0/1506776880 <== mgr.4101 v1:172.21.15.102:6816/34937 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+276324 (unknown 2965378022 0 401691773) 0x7f6c50009da0 con 0x7f6c44031160 2024-09-13T17:44:12.252 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:44:12.254 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-09-13T17:44:12.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.251+0000 7f6c68634640 1 -- 172.21.15.102:0/1506776880 >> v1:172.21.15.102:6816/34937 conn(0x7f6c44031160 legacy=0x7f6c44033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:12.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.251+0000 7f6c68634640 1 -- 172.21.15.102:0/1506776880 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6c60128330 msgr2=0x7f6c60114e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:12.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.251+0000 7f6c68634640 1 --2- 172.21.15.102:0/1506776880 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6c60128330 0x7f6c60114e80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c0100f0 tx=0x7f6c5c00f040 comp rx=0 tx=0).stop 2024-09-13T17:44:12.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.251+0000 7f6c68634640 1 -- 172.21.15.102:0/1506776880 shutdown_connections 2024-09-13T17:44:12.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.251+0000 7f6c68634640 1 --2- 172.21.15.102:0/1506776880 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6c6012bd80 0x7f6c60115900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.251+0000 7f6c68634640 1 --2- 172.21.15.102:0/1506776880 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c60128c40 0x7f6c601153c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.251+0000 7f6c68634640 1 --2- 172.21.15.102:0/1506776880 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6c60128330 0x7f6c60114e80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.251+0000 7f6c68634640 1 -- 172.21.15.102:0/1506776880 >> 172.21.15.102:0/1506776880 conn(0x7f6c6009e820 msgr2=0x7f6c60129740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:12.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.251+0000 7f6c68634640 1 -- 172.21.15.102:0/1506776880 shutdown_connections 2024-09-13T17:44:12.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.251+0000 7f6c68634640 1 -- 172.21.15.102:0/1506776880 wait complete. 2024-09-13T17:44:12.263 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2024-09-13T17:44:10.440476+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592725,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":125,"num_write_kb":1121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":93,"ondisk_log_size":93,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":93,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1191648,"kb_used_data":2212,"kb_used_omap":6,"kb_used_meta":1189369,"kb_avail":376295712,"statfs":{"total":386547056640,"available":385326809088,"internally_reserved":0,"allocated":2265088,"data_stored":1561458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6816,"internal_metadata":1217914208},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000814"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.096521+0000","last_change":"2024-09-13T17:44:09.096521+0000","last_active":"2024-09-13T17:44:09.096521+0000","last_peered":"2024-09-13T17:44:09.096521+0000","last_clean":"2024-09-13T17:44:09.096521+0000","last_became_active":"2024-09-13T17:44:09.096289+0000","last_became_peered":"2024-09-13T17:44:09.096289+0000","last_unstale":"2024-09-13T17:44:09.096521+0000","last_undegraded":"2024-09-13T17:44:09.096521+0000","last_fullsized":"2024-09-13T17:44:09.096521+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:21.494093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.100887+0000","last_change":"2024-09-13T17:44:09.100887+0000","last_active":"2024-09-13T17:44:09.100887+0000","last_peered":"2024-09-13T17:44:09.100887+0000","last_clean":"2024-09-13T17:44:09.100887+0000","last_became_active":"2024-09-13T17:44:09.100407+0000","last_became_peered":"2024-09-13T17:44:09.100407+0000","last_unstale":"2024-09-13T17:44:09.100887+0000","last_undegraded":"2024-09-13T17:44:09.100887+0000","last_fullsized":"2024-09-13T17:44:09.100887+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:00:01.667786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.084879+0000","last_change":"2024-09-13T17:44:09.084879+0000","last_active":"2024-09-13T17:44:09.084879+0000","last_peered":"2024-09-13T17:44:09.084879+0000","last_clean":"2024-09-13T17:44:09.084879+0000","last_became_active":"2024-09-13T17:44:09.084643+0000","last_became_peered":"2024-09-13T17:44:09.084643+0000","last_unstale":"2024-09-13T17:44:09.084879+0000","last_undegraded":"2024-09-13T17:44:09.084879+0000","last_fullsized":"2024-09-13T17:44:09.084879+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:41:47.399207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.105480+0000","last_change":"2024-09-13T17:44:09.105480+0000","last_active":"2024-09-13T17:44:09.105480+0000","last_peered":"2024-09-13T17:44:09.105480+0000","last_clean":"2024-09-13T17:44:09.105480+0000","last_became_active":"2024-09-13T17:44:09.105242+0000","last_became_peered":"2024-09-13T17:44:09.105242+0000","last_unstale":"2024-09-13T17:44:09.105480+0000","last_undegraded":"2024-09-13T17:44:09.105480+0000","last_fullsized":"2024-09-13T17:44:09.105480+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:17:45.470130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.096825+0000","last_change":"2024-09-13T17:44:09.096825+0000","last_active":"2024-09-13T17:44:09.096825+0000","last_peered":"2024-09-13T17:44:09.096825+0000","last_clean":"2024-09-13T17:44:09.096825+0000","last_became_active":"2024-09-13T17:44:09.096589+0000","last_became_peered":"2024-09-13T17:44:09.096589+0000","last_unstale":"2024-09-13T17:44:09.096825+0000","last_undegraded":"2024-09-13T17:44:09.096825+0000","last_fullsized":"2024-09-13T17:44:09.096825+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:46.797545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.082737+0000","last_change":"2024-09-13T17:44:09.082737+0000","last_active":"2024-09-13T17:44:09.082737+0000","last_peered":"2024-09-13T17:44:09.082737+0000","last_clean":"2024-09-13T17:44:09.082737+0000","last_became_active":"2024-09-13T17:44:09.082447+0000","last_became_peered":"2024-09-13T17:44:09.082447+0000","last_unstale":"2024-09-13T17:44:09.082737+0000","last_undegraded":"2024-09-13T17:44:09.082737+0000","last_fullsized":"2024-09-13T17:44:09.082737+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:18:08.511918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.076491+0000","last_change":"2024-09-13T17:44:09.076491+0000","last_active":"2024-09-13T17:44:09.076491+0000","last_peered":"2024-09-13T17:44:09.076491+0000","last_clean":"2024-09-13T17:44:09.076491+0000","last_became_active":"2024-09-13T17:44:09.076262+0000","last_became_peered":"2024-09-13T17:44:09.076262+0000","last_unstale":"2024-09-13T17:44:09.076491+0000","last_undegraded":"2024-09-13T17:44:09.076491+0000","last_fullsized":"2024-09-13T17:44:09.076491+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:46.797545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.081612+0000","last_change":"2024-09-13T17:44:09.081612+0000","last_active":"2024-09-13T17:44:09.081612+0000","last_peered":"2024-09-13T17:44:09.081612+0000","last_clean":"2024-09-13T17:44:09.081612+0000","last_became_active":"2024-09-13T17:44:09.081341+0000","last_became_peered":"2024-09-13T17:44:09.081341+0000","last_unstale":"2024-09-13T17:44:09.081612+0000","last_undegraded":"2024-09-13T17:44:09.081612+0000","last_fullsized":"2024-09-13T17:44:09.081612+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:19:45.314545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.083298+0000","last_change":"2024-09-13T17:44:09.083298+0000","last_active":"2024-09-13T17:44:09.083298+0000","last_peered":"2024-09-13T17:44:09.083298+0000","last_clean":"2024-09-13T17:44:09.083298+0000","last_became_active":"2024-09-13T17:44:09.083029+0000","last_became_peered":"2024-09-13T17:44:09.083029+0000","last_unstale":"2024-09-13T17:44:09.083298+0000","last_undegraded":"2024-09-13T17:44:09.083298+0000","last_fullsized":"2024-09-13T17:44:09.083298+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:48.852877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.069341+0000","last_change":"2024-09-13T17:44:09.069341+0000","last_active":"2024-09-13T17:44:09.069341+0000","last_peered":"2024-09-13T17:44:09.069341+0000","last_clean":"2024-09-13T17:44:09.069341+0000","last_became_active":"2024-09-13T17:44:09.069095+0000","last_became_peered":"2024-09-13T17:44:09.069095+0000","last_unstale":"2024-09-13T17:44:09.069341+0000","last_undegraded":"2024-09-13T17:44:09.069341+0000","last_fullsized":"2024-09-13T17:44:09.069341+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:40:06.063189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.078841+0000","last_change":"2024-09-13T17:44:09.078841+0000","last_active":"2024-09-13T17:44:09.078841+0000","last_peered":"2024-09-13T17:44:09.078841+0000","last_clean":"2024-09-13T17:44:09.078841+0000","last_became_active":"2024-09-13T17:44:09.078254+0000","last_became_peered":"2024-09-13T17:44:09.078254+0000","last_unstale":"2024-09-13T17:44:09.078841+0000","last_undegraded":"2024-09-13T17:44:09.078841+0000","last_fullsized":"2024-09-13T17:44:09.078841+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:23:33.695580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.085491+0000","last_change":"2024-09-13T17:44:09.085491+0000","last_active":"2024-09-13T17:44:09.085491+0000","last_peered":"2024-09-13T17:44:09.085491+0000","last_clean":"2024-09-13T17:44:09.085491+0000","last_became_active":"2024-09-13T17:44:09.085172+0000","last_became_peered":"2024-09-13T17:44:09.085172+0000","last_unstale":"2024-09-13T17:44:09.085491+0000","last_undegraded":"2024-09-13T17:44:09.085491+0000","last_fullsized":"2024-09-13T17:44:09.085491+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:40:06.063189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.087234+0000","last_change":"2024-09-13T17:44:09.087234+0000","last_active":"2024-09-13T17:44:09.087234+0000","last_peered":"2024-09-13T17:44:09.087234+0000","last_clean":"2024-09-13T17:44:09.087234+0000","last_became_active":"2024-09-13T17:44:09.086891+0000","last_became_peered":"2024-09-13T17:44:09.086891+0000","last_unstale":"2024-09-13T17:44:09.087234+0000","last_undegraded":"2024-09-13T17:44:09.087234+0000","last_fullsized":"2024-09-13T17:44:09.087234+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:38.841320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.100794+0000","last_change":"2024-09-13T17:44:09.100794+0000","last_active":"2024-09-13T17:44:09.100794+0000","last_peered":"2024-09-13T17:44:09.100794+0000","last_clean":"2024-09-13T17:44:09.100794+0000","last_became_active":"2024-09-13T17:44:09.100575+0000","last_became_peered":"2024-09-13T17:44:09.100575+0000","last_unstale":"2024-09-13T17:44:09.100794+0000","last_undegraded":"2024-09-13T17:44:09.100794+0000","last_fullsized":"2024-09-13T17:44:09.100794+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:38.841320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.101130+0000","last_change":"2024-09-13T17:44:09.101130+0000","last_active":"2024-09-13T17:44:09.101130+0000","last_peered":"2024-09-13T17:44:09.101130+0000","last_clean":"2024-09-13T17:44:09.101130+0000","last_became_active":"2024-09-13T17:44:09.100865+0000","last_became_peered":"2024-09-13T17:44:09.100865+0000","last_unstale":"2024-09-13T17:44:09.101130+0000","last_undegraded":"2024-09-13T17:44:09.101130+0000","last_fullsized":"2024-09-13T17:44:09.101130+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:53:41.289117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.076274+0000","last_change":"2024-09-13T17:44:09.076274+0000","last_active":"2024-09-13T17:44:09.076274+0000","last_peered":"2024-09-13T17:44:09.076274+0000","last_clean":"2024-09-13T17:44:09.076274+0000","last_became_active":"2024-09-13T17:44:09.075782+0000","last_became_peered":"2024-09-13T17:44:09.075782+0000","last_unstale":"2024-09-13T17:44:09.076274+0000","last_undegraded":"2024-09-13T17:44:09.076274+0000","last_fullsized":"2024-09-13T17:44:09.076274+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:23:33.695580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.095089+0000","last_change":"2024-09-13T17:44:09.095089+0000","last_active":"2024-09-13T17:44:09.095089+0000","last_peered":"2024-09-13T17:44:09.095089+0000","last_clean":"2024-09-13T17:44:09.095089+0000","last_became_active":"2024-09-13T17:44:09.094856+0000","last_became_peered":"2024-09-13T17:44:09.094856+0000","last_unstale":"2024-09-13T17:44:09.095089+0000","last_undegraded":"2024-09-13T17:44:09.095089+0000","last_fullsized":"2024-09-13T17:44:09.095089+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:52.363453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.095140+0000","last_change":"2024-09-13T17:44:09.095140+0000","last_active":"2024-09-13T17:44:09.095140+0000","last_peered":"2024-09-13T17:44:09.095140+0000","last_clean":"2024-09-13T17:44:09.095140+0000","last_became_active":"2024-09-13T17:44:09.094922+0000","last_became_peered":"2024-09-13T17:44:09.094922+0000","last_unstale":"2024-09-13T17:44:09.095140+0000","last_undegraded":"2024-09-13T17:44:09.095140+0000","last_fullsized":"2024-09-13T17:44:09.095140+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:07.163678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.097429+0000","last_change":"2024-09-13T17:44:09.097429+0000","last_active":"2024-09-13T17:44:09.097429+0000","last_peered":"2024-09-13T17:44:09.097429+0000","last_clean":"2024-09-13T17:44:09.097429+0000","last_became_active":"2024-09-13T17:44:09.097178+0000","last_became_peered":"2024-09-13T17:44:09.097178+0000","last_unstale":"2024-09-13T17:44:09.097429+0000","last_undegraded":"2024-09-13T17:44:09.097429+0000","last_fullsized":"2024-09-13T17:44:09.097429+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:30:37.262970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.083241+0000","last_change":"2024-09-13T17:44:09.083241+0000","last_active":"2024-09-13T17:44:09.083241+0000","last_peered":"2024-09-13T17:44:09.083241+0000","last_clean":"2024-09-13T17:44:09.083241+0000","last_became_active":"2024-09-13T17:44:09.082984+0000","last_became_peered":"2024-09-13T17:44:09.082984+0000","last_unstale":"2024-09-13T17:44:09.083241+0000","last_undegraded":"2024-09-13T17:44:09.083241+0000","last_fullsized":"2024-09-13T17:44:09.083241+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:00:07.720440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.077433+0000","last_change":"2024-09-13T17:44:09.077433+0000","last_active":"2024-09-13T17:44:09.077433+0000","last_peered":"2024-09-13T17:44:09.077433+0000","last_clean":"2024-09-13T17:44:09.077433+0000","last_became_active":"2024-09-13T17:44:09.076107+0000","last_became_peered":"2024-09-13T17:44:09.076107+0000","last_unstale":"2024-09-13T17:44:09.077433+0000","last_undegraded":"2024-09-13T17:44:09.077433+0000","last_fullsized":"2024-09-13T17:44:09.077433+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:10:25.431639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.086733+0000","last_change":"2024-09-13T17:44:09.086733+0000","last_active":"2024-09-13T17:44:09.086733+0000","last_peered":"2024-09-13T17:44:09.086733+0000","last_clean":"2024-09-13T17:44:09.086733+0000","last_became_active":"2024-09-13T17:44:09.086214+0000","last_became_peered":"2024-09-13T17:44:09.086214+0000","last_unstale":"2024-09-13T17:44:09.086733+0000","last_undegraded":"2024-09-13T17:44:09.086733+0000","last_fullsized":"2024-09-13T17:44:09.086733+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:29.109889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.100766+0000","last_change":"2024-09-13T17:44:09.100766+0000","last_active":"2024-09-13T17:44:09.100766+0000","last_peered":"2024-09-13T17:44:09.100766+0000","last_clean":"2024-09-13T17:44:09.100766+0000","last_became_active":"2024-09-13T17:44:09.100532+0000","last_became_peered":"2024-09-13T17:44:09.100532+0000","last_unstale":"2024-09-13T17:44:09.100766+0000","last_undegraded":"2024-09-13T17:44:09.100766+0000","last_fullsized":"2024-09-13T17:44:09.100766+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:23:20.557832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.082870+0000","last_change":"2024-09-13T17:44:09.082870+0000","last_active":"2024-09-13T17:44:09.082870+0000","last_peered":"2024-09-13T17:44:09.082870+0000","last_clean":"2024-09-13T17:44:09.082870+0000","last_became_active":"2024-09-13T17:44:09.082641+0000","last_became_peered":"2024-09-13T17:44:09.082641+0000","last_unstale":"2024-09-13T17:44:09.082870+0000","last_undegraded":"2024-09-13T17:44:09.082870+0000","last_fullsized":"2024-09-13T17:44:09.082870+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:23:36.075940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.097350+0000","last_change":"2024-09-13T17:44:09.097350+0000","last_active":"2024-09-13T17:44:09.097350+0000","last_peered":"2024-09-13T17:44:09.097350+0000","last_clean":"2024-09-13T17:44:09.097350+0000","last_became_active":"2024-09-13T17:44:09.097114+0000","last_became_peered":"2024-09-13T17:44:09.097114+0000","last_unstale":"2024-09-13T17:44:09.097350+0000","last_undegraded":"2024-09-13T17:44:09.097350+0000","last_fullsized":"2024-09-13T17:44:09.097350+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:10:25.431639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.085005+0000","last_change":"2024-09-13T17:44:09.085005+0000","last_active":"2024-09-13T17:44:09.085005+0000","last_peered":"2024-09-13T17:44:09.085005+0000","last_clean":"2024-09-13T17:44:09.085005+0000","last_became_active":"2024-09-13T17:44:09.084503+0000","last_became_peered":"2024-09-13T17:44:09.084503+0000","last_unstale":"2024-09-13T17:44:09.085005+0000","last_undegraded":"2024-09-13T17:44:09.085005+0000","last_fullsized":"2024-09-13T17:44:09.085005+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:30:37.262970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.085567+0000","last_change":"2024-09-13T17:44:09.085567+0000","last_active":"2024-09-13T17:44:09.085567+0000","last_peered":"2024-09-13T17:44:09.085567+0000","last_clean":"2024-09-13T17:44:09.085567+0000","last_became_active":"2024-09-13T17:44:09.085234+0000","last_became_peered":"2024-09-13T17:44:09.085234+0000","last_unstale":"2024-09-13T17:44:09.085567+0000","last_undegraded":"2024-09-13T17:44:09.085567+0000","last_fullsized":"2024-09-13T17:44:09.085567+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:43.983384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.082703+0000","last_change":"2024-09-13T17:44:09.082703+0000","last_active":"2024-09-13T17:44:09.082703+0000","last_peered":"2024-09-13T17:44:09.082703+0000","last_clean":"2024-09-13T17:44:09.082703+0000","last_became_active":"2024-09-13T17:44:09.082474+0000","last_became_peered":"2024-09-13T17:44:09.082474+0000","last_unstale":"2024-09-13T17:44:09.082703+0000","last_undegraded":"2024-09-13T17:44:09.082703+0000","last_fullsized":"2024-09-13T17:44:09.082703+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:44.944657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.082198+0000","last_change":"2024-09-13T17:44:09.082198+0000","last_active":"2024-09-13T17:44:09.082198+0000","last_peered":"2024-09-13T17:44:09.082198+0000","last_clean":"2024-09-13T17:44:09.082198+0000","last_became_active":"2024-09-13T17:44:09.081672+0000","last_became_peered":"2024-09-13T17:44:09.081672+0000","last_unstale":"2024-09-13T17:44:09.082198+0000","last_undegraded":"2024-09-13T17:44:09.082198+0000","last_fullsized":"2024-09-13T17:44:09.082198+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:15:17.170944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.084099+0000","last_change":"2024-09-13T17:44:09.084099+0000","last_active":"2024-09-13T17:44:09.084099+0000","last_peered":"2024-09-13T17:44:09.084099+0000","last_clean":"2024-09-13T17:44:09.084099+0000","last_became_active":"2024-09-13T17:44:09.082155+0000","last_became_peered":"2024-09-13T17:44:09.082155+0000","last_unstale":"2024-09-13T17:44:09.084099+0000","last_undegraded":"2024-09-13T17:44:09.084099+0000","last_fullsized":"2024-09-13T17:44:09.084099+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:08:37.816520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.086616+0000","last_change":"2024-09-13T17:44:09.086616+0000","last_active":"2024-09-13T17:44:09.086616+0000","last_peered":"2024-09-13T17:44:09.086616+0000","last_clean":"2024-09-13T17:44:09.086616+0000","last_became_active":"2024-09-13T17:44:09.085504+0000","last_became_peered":"2024-09-13T17:44:09.085504+0000","last_unstale":"2024-09-13T17:44:09.086616+0000","last_undegraded":"2024-09-13T17:44:09.086616+0000","last_fullsized":"2024-09-13T17:44:09.086616+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:43.983384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.085649+0000","last_change":"2024-09-13T17:44:09.085649+0000","last_active":"2024-09-13T17:44:09.085649+0000","last_peered":"2024-09-13T17:44:09.085649+0000","last_clean":"2024-09-13T17:44:09.085649+0000","last_became_active":"2024-09-13T17:44:09.085422+0000","last_became_peered":"2024-09-13T17:44:09.085422+0000","last_unstale":"2024-09-13T17:44:09.085649+0000","last_undegraded":"2024-09-13T17:44:09.085649+0000","last_fullsized":"2024-09-13T17:44:09.085649+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:17:45.470130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.090868+0000","last_change":"2024-09-13T17:44:09.090868+0000","last_active":"2024-09-13T17:44:09.090868+0000","last_peered":"2024-09-13T17:44:09.090868+0000","last_clean":"2024-09-13T17:44:09.090868+0000","last_became_active":"2024-09-13T17:44:09.090651+0000","last_became_peered":"2024-09-13T17:44:09.090651+0000","last_unstale":"2024-09-13T17:44:09.090868+0000","last_undegraded":"2024-09-13T17:44:09.090868+0000","last_fullsized":"2024-09-13T17:44:09.090868+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:59:17.503064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"31'4","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.103927+0000","last_change":"2024-09-13T17:44:08.078924+0000","last_active":"2024-09-13T17:44:09.103927+0000","last_peered":"2024-09-13T17:44:09.103927+0000","last_clean":"2024-09-13T17:44:09.103927+0000","last_became_active":"2024-09-13T17:44:08.078686+0000","last_became_peered":"2024-09-13T17:44:08.078686+0000","last_unstale":"2024-09-13T17:44:09.103927+0000","last_undegraded":"2024-09-13T17:44:09.103927+0000","last_fullsized":"2024-09-13T17:44:09.103927+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:00:18.297348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.082059+0000","last_change":"2024-09-13T17:44:09.082059+0000","last_active":"2024-09-13T17:44:09.082059+0000","last_peered":"2024-09-13T17:44:09.082059+0000","last_clean":"2024-09-13T17:44:09.082059+0000","last_became_active":"2024-09-13T17:44:09.080468+0000","last_became_peered":"2024-09-13T17:44:09.080468+0000","last_unstale":"2024-09-13T17:44:09.082059+0000","last_undegraded":"2024-09-13T17:44:09.082059+0000","last_fullsized":"2024-09-13T17:44:09.082059+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:00:07.720440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"31'1","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.066141+0000","last_change":"2024-09-13T17:44:08.087199+0000","last_active":"2024-09-13T17:44:09.066141+0000","last_peered":"2024-09-13T17:44:09.066141+0000","last_clean":"2024-09-13T17:44:09.066141+0000","last_became_active":"2024-09-13T17:44:08.086469+0000","last_became_peered":"2024-09-13T17:44:08.086469+0000","last_unstale":"2024-09-13T17:44:09.066141+0000","last_undegraded":"2024-09-13T17:44:09.066141+0000","last_fullsized":"2024-09-13T17:44:09.066141+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:45.333857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.100981+0000","last_change":"2024-09-13T17:44:09.100981+0000","last_active":"2024-09-13T17:44:09.100981+0000","last_peered":"2024-09-13T17:44:09.100981+0000","last_clean":"2024-09-13T17:44:09.100981+0000","last_became_active":"2024-09-13T17:44:09.100713+0000","last_became_peered":"2024-09-13T17:44:09.100713+0000","last_unstale":"2024-09-13T17:44:09.100981+0000","last_undegraded":"2024-09-13T17:44:09.100981+0000","last_fullsized":"2024-09-13T17:44:09.100981+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:30:59.005533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.094766+0000","last_change":"2024-09-13T17:44:08.085058+0000","last_active":"2024-09-13T17:44:09.094766+0000","last_peered":"2024-09-13T17:44:09.094766+0000","last_clean":"2024-09-13T17:44:09.094766+0000","last_became_active":"2024-09-13T17:44:08.084831+0000","last_became_peered":"2024-09-13T17:44:08.084831+0000","last_unstale":"2024-09-13T17:44:09.094766+0000","last_undegraded":"2024-09-13T17:44:09.094766+0000","last_fullsized":"2024-09-13T17:44:09.094766+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:31.939247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.087821+0000","last_change":"2024-09-13T17:44:09.087821+0000","last_active":"2024-09-13T17:44:09.087821+0000","last_peered":"2024-09-13T17:44:09.087821+0000","last_clean":"2024-09-13T17:44:09.087821+0000","last_became_active":"2024-09-13T17:44:09.087579+0000","last_became_peered":"2024-09-13T17:44:09.087579+0000","last_unstale":"2024-09-13T17:44:09.087821+0000","last_undegraded":"2024-09-13T17:44:09.087821+0000","last_fullsized":"2024-09-13T17:44:09.087821+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:03.454864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.106047+0000","last_change":"2024-09-13T17:44:08.079732+0000","last_active":"2024-09-13T17:44:09.106047+0000","last_peered":"2024-09-13T17:44:09.106047+0000","last_clean":"2024-09-13T17:44:09.106047+0000","last_became_active":"2024-09-13T17:44:08.079375+0000","last_became_peered":"2024-09-13T17:44:08.079375+0000","last_unstale":"2024-09-13T17:44:09.106047+0000","last_undegraded":"2024-09-13T17:44:09.106047+0000","last_fullsized":"2024-09-13T17:44:09.106047+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:16:25.243221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.078321+0000","last_change":"2024-09-13T17:44:09.078321+0000","last_active":"2024-09-13T17:44:09.078321+0000","last_peered":"2024-09-13T17:44:09.078321+0000","last_clean":"2024-09-13T17:44:09.078321+0000","last_became_active":"2024-09-13T17:44:09.078085+0000","last_became_peered":"2024-09-13T17:44:09.078085+0000","last_unstale":"2024-09-13T17:44:09.078321+0000","last_undegraded":"2024-09-13T17:44:09.078321+0000","last_fullsized":"2024-09-13T17:44:09.078321+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:08:00.966936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"31'1","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.074489+0000","last_change":"2024-09-13T17:44:08.087304+0000","last_active":"2024-09-13T17:44:09.074489+0000","last_peered":"2024-09-13T17:44:09.074489+0000","last_clean":"2024-09-13T17:44:09.074489+0000","last_became_active":"2024-09-13T17:44:08.086913+0000","last_became_peered":"2024-09-13T17:44:08.086913+0000","last_unstale":"2024-09-13T17:44:09.074489+0000","last_undegraded":"2024-09-13T17:44:09.074489+0000","last_fullsized":"2024-09-13T17:44:09.074489+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:06:31.498523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.078727+0000","last_change":"2024-09-13T17:44:09.078727+0000","last_active":"2024-09-13T17:44:09.078727+0000","last_peered":"2024-09-13T17:44:09.078727+0000","last_clean":"2024-09-13T17:44:09.078727+0000","last_became_active":"2024-09-13T17:44:09.078337+0000","last_became_peered":"2024-09-13T17:44:09.078337+0000","last_unstale":"2024-09-13T17:44:09.078727+0000","last_undegraded":"2024-09-13T17:44:09.078727+0000","last_fullsized":"2024-09-13T17:44:09.078727+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:30:37.262970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"31'1","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.062588+0000","last_change":"2024-09-13T17:44:08.088763+0000","last_active":"2024-09-13T17:44:09.062588+0000","last_peered":"2024-09-13T17:44:09.062588+0000","last_clean":"2024-09-13T17:44:09.062588+0000","last_became_active":"2024-09-13T17:44:08.088465+0000","last_became_peered":"2024-09-13T17:44:08.088465+0000","last_unstale":"2024-09-13T17:44:09.062588+0000","last_undegraded":"2024-09-13T17:44:09.062588+0000","last_fullsized":"2024-09-13T17:44:09.062588+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:45.333857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.086254+0000","last_change":"2024-09-13T17:44:09.086254+0000","last_active":"2024-09-13T17:44:09.086254+0000","last_peered":"2024-09-13T17:44:09.086254+0000","last_clean":"2024-09-13T17:44:09.086254+0000","last_became_active":"2024-09-13T17:44:09.084768+0000","last_became_peered":"2024-09-13T17:44:09.084768+0000","last_unstale":"2024-09-13T17:44:09.086254+0000","last_undegraded":"2024-09-13T17:44:09.086254+0000","last_fullsized":"2024-09-13T17:44:09.086254+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:34:38.550964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.089244+0000","last_change":"2024-09-13T17:44:08.080175+0000","last_active":"2024-09-13T17:44:08.089244+0000","last_peered":"2024-09-13T17:44:08.089244+0000","last_clean":"2024-09-13T17:44:08.089244+0000","last_became_active":"2024-09-13T17:44:08.079917+0000","last_became_peered":"2024-09-13T17:44:08.079917+0000","last_unstale":"2024-09-13T17:44:08.089244+0000","last_undegraded":"2024-09-13T17:44:08.089244+0000","last_fullsized":"2024-09-13T17:44:08.089244+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:11:10.137498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.092808+0000","last_change":"2024-09-13T17:44:09.092808+0000","last_active":"2024-09-13T17:44:09.092808+0000","last_peered":"2024-09-13T17:44:09.092808+0000","last_clean":"2024-09-13T17:44:09.092808+0000","last_became_active":"2024-09-13T17:44:09.092341+0000","last_became_peered":"2024-09-13T17:44:09.092341+0000","last_unstale":"2024-09-13T17:44:09.092808+0000","last_undegraded":"2024-09-13T17:44:09.092808+0000","last_fullsized":"2024-09-13T17:44:09.092808+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:26:57.016183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.089407+0000","last_change":"2024-09-13T17:44:08.083995+0000","last_active":"2024-09-13T17:44:08.089407+0000","last_peered":"2024-09-13T17:44:08.089407+0000","last_clean":"2024-09-13T17:44:08.089407+0000","last_became_active":"2024-09-13T17:44:08.083754+0000","last_became_peered":"2024-09-13T17:44:08.083754+0000","last_unstale":"2024-09-13T17:44:08.089407+0000","last_undegraded":"2024-09-13T17:44:08.089407+0000","last_fullsized":"2024-09-13T17:44:08.089407+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:10.712527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.105073+0000","last_change":"2024-09-13T17:44:09.105073+0000","last_active":"2024-09-13T17:44:09.105073+0000","last_peered":"2024-09-13T17:44:09.105073+0000","last_clean":"2024-09-13T17:44:09.105073+0000","last_became_active":"2024-09-13T17:44:09.104883+0000","last_became_peered":"2024-09-13T17:44:09.104883+0000","last_unstale":"2024-09-13T17:44:09.105073+0000","last_undegraded":"2024-09-13T17:44:09.105073+0000","last_fullsized":"2024-09-13T17:44:09.105073+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:43.983384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.083050+0000","last_change":"2024-09-13T17:44:08.081802+0000","last_active":"2024-09-13T17:44:08.083050+0000","last_peered":"2024-09-13T17:44:08.083050+0000","last_clean":"2024-09-13T17:44:08.083050+0000","last_became_active":"2024-09-13T17:44:08.081561+0000","last_became_peered":"2024-09-13T17:44:08.081561+0000","last_unstale":"2024-09-13T17:44:08.083050+0000","last_undegraded":"2024-09-13T17:44:08.083050+0000","last_fullsized":"2024-09-13T17:44:08.083050+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:00:06.716152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.100180+0000","last_change":"2024-09-13T17:44:09.100180+0000","last_active":"2024-09-13T17:44:09.100180+0000","last_peered":"2024-09-13T17:44:09.100180+0000","last_clean":"2024-09-13T17:44:09.100180+0000","last_became_active":"2024-09-13T17:44:09.097650+0000","last_became_peered":"2024-09-13T17:44:09.097650+0000","last_unstale":"2024-09-13T17:44:09.100180+0000","last_undegraded":"2024-09-13T17:44:09.100180+0000","last_fullsized":"2024-09-13T17:44:09.100180+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:08:00.966936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.082752+0000","last_change":"2024-09-13T17:44:08.080765+0000","last_active":"2024-09-13T17:44:08.082752+0000","last_peered":"2024-09-13T17:44:08.082752+0000","last_clean":"2024-09-13T17:44:08.082752+0000","last_became_active":"2024-09-13T17:44:08.080523+0000","last_became_peered":"2024-09-13T17:44:08.080523+0000","last_unstale":"2024-09-13T17:44:08.082752+0000","last_undegraded":"2024-09-13T17:44:08.082752+0000","last_fullsized":"2024-09-13T17:44:08.082752+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:10:24.427351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.077080+0000","last_change":"2024-09-13T17:44:09.077080+0000","last_active":"2024-09-13T17:44:09.077080+0000","last_peered":"2024-09-13T17:44:09.077080+0000","last_clean":"2024-09-13T17:44:09.077080+0000","last_became_active":"2024-09-13T17:44:09.076529+0000","last_became_peered":"2024-09-13T17:44:09.076529+0000","last_unstale":"2024-09-13T17:44:09.077080+0000","last_undegraded":"2024-09-13T17:44:09.077080+0000","last_fullsized":"2024-09-13T17:44:09.077080+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:38.841320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"31'3","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.099802+0000","last_change":"2024-09-13T17:44:08.080977+0000","last_active":"2024-09-13T17:44:09.099802+0000","last_peered":"2024-09-13T17:44:09.099802+0000","last_clean":"2024-09-13T17:44:09.099802+0000","last_became_active":"2024-09-13T17:44:08.080717+0000","last_became_peered":"2024-09-13T17:44:08.080717+0000","last_unstale":"2024-09-13T17:44:09.099802+0000","last_undegraded":"2024-09-13T17:44:09.099802+0000","last_fullsized":"2024-09-13T17:44:09.099802+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:40:05.058901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.085550+0000","last_change":"2024-09-13T17:44:09.085550+0000","last_active":"2024-09-13T17:44:09.085550+0000","last_peered":"2024-09-13T17:44:09.085550+0000","last_clean":"2024-09-13T17:44:09.085550+0000","last_became_active":"2024-09-13T17:44:09.085317+0000","last_became_peered":"2024-09-13T17:44:09.085317+0000","last_unstale":"2024-09-13T17:44:09.085550+0000","last_undegraded":"2024-09-13T17:44:09.085550+0000","last_fullsized":"2024-09-13T17:44:09.085550+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:10:25.431639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"31'6","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.106835+0000","last_change":"2024-09-13T17:44:08.082800+0000","last_active":"2024-09-13T17:44:09.106835+0000","last_peered":"2024-09-13T17:44:09.106835+0000","last_clean":"2024-09-13T17:44:09.106835+0000","last_became_active":"2024-09-13T17:44:08.082558+0000","last_became_peered":"2024-09-13T17:44:08.082558+0000","last_unstale":"2024-09-13T17:44:09.106835+0000","last_undegraded":"2024-09-13T17:44:09.106835+0000","last_fullsized":"2024-09-13T17:44:09.106835+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:45.793256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.098333+0000","last_change":"2024-09-13T17:44:09.098333+0000","last_active":"2024-09-13T17:44:09.098333+0000","last_peered":"2024-09-13T17:44:09.098333+0000","last_clean":"2024-09-13T17:44:09.098333+0000","last_became_active":"2024-09-13T17:44:09.098092+0000","last_became_peered":"2024-09-13T17:44:09.098092+0000","last_unstale":"2024-09-13T17:44:09.098333+0000","last_undegraded":"2024-09-13T17:44:09.098333+0000","last_fullsized":"2024-09-13T17:44:09.098333+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:17:45.470130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.095273+0000","last_change":"2024-09-13T17:44:08.086760+0000","last_active":"2024-09-13T17:44:09.095273+0000","last_peered":"2024-09-13T17:44:09.095273+0000","last_clean":"2024-09-13T17:44:09.095273+0000","last_became_active":"2024-09-13T17:44:08.086516+0000","last_became_peered":"2024-09-13T17:44:08.086516+0000","last_unstale":"2024-09-13T17:44:09.095273+0000","last_undegraded":"2024-09-13T17:44:09.095273+0000","last_fullsized":"2024-09-13T17:44:09.095273+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:33:03.626924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.095841+0000","last_change":"2024-09-13T17:44:09.095841+0000","last_active":"2024-09-13T17:44:09.095841+0000","last_peered":"2024-09-13T17:44:09.095841+0000","last_clean":"2024-09-13T17:44:09.095841+0000","last_became_active":"2024-09-13T17:44:09.095603+0000","last_became_peered":"2024-09-13T17:44:09.095603+0000","last_unstale":"2024-09-13T17:44:09.095841+0000","last_undegraded":"2024-09-13T17:44:09.095841+0000","last_fullsized":"2024-09-13T17:44:09.095841+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:09:59.370569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.089292+0000","last_change":"2024-09-13T17:44:08.080876+0000","last_active":"2024-09-13T17:44:08.089292+0000","last_peered":"2024-09-13T17:44:08.089292+0000","last_clean":"2024-09-13T17:44:08.089292+0000","last_became_active":"2024-09-13T17:44:08.080073+0000","last_became_peered":"2024-09-13T17:44:08.080073+0000","last_unstale":"2024-09-13T17:44:08.089292+0000","last_undegraded":"2024-09-13T17:44:08.089292+0000","last_fullsized":"2024-09-13T17:44:08.089292+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:00:18.297348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.086890+0000","last_change":"2024-09-13T17:44:09.086890+0000","last_active":"2024-09-13T17:44:09.086890+0000","last_peered":"2024-09-13T17:44:09.086890+0000","last_clean":"2024-09-13T17:44:09.086890+0000","last_became_active":"2024-09-13T17:44:09.086512+0000","last_became_peered":"2024-09-13T17:44:09.086512+0000","last_unstale":"2024-09-13T17:44:09.086890+0000","last_undegraded":"2024-09-13T17:44:09.086890+0000","last_fullsized":"2024-09-13T17:44:09.086890+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:33:04.631213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.086152+0000","last_change":"2024-09-13T17:44:08.082647+0000","last_active":"2024-09-13T17:44:08.086152+0000","last_peered":"2024-09-13T17:44:08.086152+0000","last_clean":"2024-09-13T17:44:08.086152+0000","last_became_active":"2024-09-13T17:44:08.082414+0000","last_became_peered":"2024-09-13T17:44:08.082414+0000","last_unstale":"2024-09-13T17:44:08.086152+0000","last_undegraded":"2024-09-13T17:44:08.086152+0000","last_fullsized":"2024-09-13T17:44:08.086152+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:23:32.691291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.091976+0000","last_change":"2024-09-13T17:44:09.091976+0000","last_active":"2024-09-13T17:44:09.091976+0000","last_peered":"2024-09-13T17:44:09.091976+0000","last_clean":"2024-09-13T17:44:09.091976+0000","last_became_active":"2024-09-13T17:44:09.091714+0000","last_became_peered":"2024-09-13T17:44:09.091714+0000","last_unstale":"2024-09-13T17:44:09.091976+0000","last_undegraded":"2024-09-13T17:44:09.091976+0000","last_fullsized":"2024-09-13T17:44:09.091976+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:14.755193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"31'8","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.105106+0000","last_change":"2024-09-13T17:44:08.080734+0000","last_active":"2024-09-13T17:44:09.105106+0000","last_peered":"2024-09-13T17:44:09.105106+0000","last_clean":"2024-09-13T17:44:09.105106+0000","last_became_active":"2024-09-13T17:44:08.080497+0000","last_became_peered":"2024-09-13T17:44:08.080497+0000","last_unstale":"2024-09-13T17:44:09.105106+0000","last_undegraded":"2024-09-13T17:44:09.105106+0000","last_fullsized":"2024-09-13T17:44:09.105106+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:11:10.137498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.092611+0000","last_change":"2024-09-13T17:44:09.092611+0000","last_active":"2024-09-13T17:44:09.092611+0000","last_peered":"2024-09-13T17:44:09.092611+0000","last_clean":"2024-09-13T17:44:09.092611+0000","last_became_active":"2024-09-13T17:44:09.091079+0000","last_became_peered":"2024-09-13T17:44:09.091079+0000","last_unstale":"2024-09-13T17:44:09.092611+0000","last_undegraded":"2024-09-13T17:44:09.092611+0000","last_fullsized":"2024-09-13T17:44:09.092611+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:53:57.379081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"31'1","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.063609+0000","last_change":"2024-09-13T17:44:08.084294+0000","last_active":"2024-09-13T17:44:09.063609+0000","last_peered":"2024-09-13T17:44:09.063609+0000","last_clean":"2024-09-13T17:44:09.063609+0000","last_became_active":"2024-09-13T17:44:08.084033+0000","last_became_peered":"2024-09-13T17:44:08.084033+0000","last_unstale":"2024-09-13T17:44:09.063609+0000","last_undegraded":"2024-09-13T17:44:09.063609+0000","last_fullsized":"2024-09-13T17:44:09.063609+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:04:26.313895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.081895+0000","last_change":"2024-09-13T17:44:09.081895+0000","last_active":"2024-09-13T17:44:09.081895+0000","last_peered":"2024-09-13T17:44:09.081895+0000","last_clean":"2024-09-13T17:44:09.081895+0000","last_became_active":"2024-09-13T17:44:09.080183+0000","last_became_peered":"2024-09-13T17:44:09.080183+0000","last_unstale":"2024-09-13T17:44:09.081895+0000","last_undegraded":"2024-09-13T17:44:09.081895+0000","last_fullsized":"2024-09-13T17:44:09.081895+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:33:04.631213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.102962+0000","last_change":"2024-09-13T17:44:08.091269+0000","last_active":"2024-09-13T17:44:09.102962+0000","last_peered":"2024-09-13T17:44:09.102962+0000","last_clean":"2024-09-13T17:44:09.102962+0000","last_became_active":"2024-09-13T17:44:08.091000+0000","last_became_peered":"2024-09-13T17:44:08.091000+0000","last_unstale":"2024-09-13T17:44:09.102962+0000","last_undegraded":"2024-09-13T17:44:09.102962+0000","last_fullsized":"2024-09-13T17:44:09.102962+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:47.848588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.085165+0000","last_change":"2024-09-13T17:44:09.085165+0000","last_active":"2024-09-13T17:44:09.085165+0000","last_peered":"2024-09-13T17:44:09.085165+0000","last_clean":"2024-09-13T17:44:09.085165+0000","last_became_active":"2024-09-13T17:44:09.084779+0000","last_became_peered":"2024-09-13T17:44:09.084779+0000","last_unstale":"2024-09-13T17:44:09.085165+0000","last_undegraded":"2024-09-13T17:44:09.085165+0000","last_fullsized":"2024-09-13T17:44:09.085165+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:23:36.075940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"31'1","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.073400+0000","last_change":"2024-09-13T17:44:08.086166+0000","last_active":"2024-09-13T17:44:09.073400+0000","last_peered":"2024-09-13T17:44:09.073400+0000","last_clean":"2024-09-13T17:44:09.073400+0000","last_became_active":"2024-09-13T17:44:08.085913+0000","last_became_peered":"2024-09-13T17:44:08.085913+0000","last_unstale":"2024-09-13T17:44:09.073400+0000","last_undegraded":"2024-09-13T17:44:09.073400+0000","last_fullsized":"2024-09-13T17:44:09.073400+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:17:08.989245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.101360+0000","last_change":"2024-09-13T17:44:09.101360+0000","last_active":"2024-09-13T17:44:09.101360+0000","last_peered":"2024-09-13T17:44:09.101360+0000","last_clean":"2024-09-13T17:44:09.101360+0000","last_became_active":"2024-09-13T17:44:09.101119+0000","last_became_peered":"2024-09-13T17:44:09.101119+0000","last_unstale":"2024-09-13T17:44:09.101360+0000","last_undegraded":"2024-09-13T17:44:09.101360+0000","last_fullsized":"2024-09-13T17:44:09.101360+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:18:08.511918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.092056+0000","last_change":"2024-09-13T17:44:08.084464+0000","last_active":"2024-09-13T17:44:08.092056+0000","last_peered":"2024-09-13T17:44:08.092056+0000","last_clean":"2024-09-13T17:44:08.092056+0000","last_became_active":"2024-09-13T17:44:08.084233+0000","last_became_peered":"2024-09-13T17:44:08.084233+0000","last_unstale":"2024-09-13T17:44:08.092056+0000","last_undegraded":"2024-09-13T17:44:08.092056+0000","last_fullsized":"2024-09-13T17:44:08.092056+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:15.312171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.091582+0000","last_change":"2024-09-13T17:44:09.091582+0000","last_active":"2024-09-13T17:44:09.091582+0000","last_peered":"2024-09-13T17:44:09.091582+0000","last_clean":"2024-09-13T17:44:09.091582+0000","last_became_active":"2024-09-13T17:44:09.091358+0000","last_became_peered":"2024-09-13T17:44:09.091358+0000","last_unstale":"2024-09-13T17:44:09.091582+0000","last_undegraded":"2024-09-13T17:44:09.091582+0000","last_fullsized":"2024-09-13T17:44:09.091582+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:51:47.981712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.104879+0000","last_change":"2024-09-13T17:44:08.087156+0000","last_active":"2024-09-13T17:44:09.104879+0000","last_peered":"2024-09-13T17:44:09.104879+0000","last_clean":"2024-09-13T17:44:09.104879+0000","last_became_active":"2024-09-13T17:44:08.086617+0000","last_became_peered":"2024-09-13T17:44:08.086617+0000","last_unstale":"2024-09-13T17:44:09.104879+0000","last_undegraded":"2024-09-13T17:44:09.104879+0000","last_fullsized":"2024-09-13T17:44:09.104879+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:01:51.213503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.078569+0000","last_change":"2024-09-13T17:44:09.078569+0000","last_active":"2024-09-13T17:44:09.078569+0000","last_peered":"2024-09-13T17:44:09.078569+0000","last_clean":"2024-09-13T17:44:09.078569+0000","last_became_active":"2024-09-13T17:44:09.078028+0000","last_became_peered":"2024-09-13T17:44:09.078028+0000","last_unstale":"2024-09-13T17:44:09.078569+0000","last_undegraded":"2024-09-13T17:44:09.078569+0000","last_fullsized":"2024-09-13T17:44:09.078569+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:00.260017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.092168+0000","last_change":"2024-09-13T17:44:08.090522+0000","last_active":"2024-09-13T17:44:08.092168+0000","last_peered":"2024-09-13T17:44:08.092168+0000","last_clean":"2024-09-13T17:44:08.092168+0000","last_became_active":"2024-09-13T17:44:08.090231+0000","last_became_peered":"2024-09-13T17:44:08.090231+0000","last_unstale":"2024-09-13T17:44:08.092168+0000","last_undegraded":"2024-09-13T17:44:08.092168+0000","last_fullsized":"2024-09-13T17:44:08.092168+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:48.658048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.076366+0000","last_change":"2024-09-13T17:44:09.076366+0000","last_active":"2024-09-13T17:44:09.076366+0000","last_peered":"2024-09-13T17:44:09.076366+0000","last_clean":"2024-09-13T17:44:09.076366+0000","last_became_active":"2024-09-13T17:44:09.075083+0000","last_became_peered":"2024-09-13T17:44:09.075083+0000","last_unstale":"2024-09-13T17:44:09.076366+0000","last_undegraded":"2024-09-13T17:44:09.076366+0000","last_fullsized":"2024-09-13T17:44:09.076366+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:18:08.511918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.089264+0000","last_change":"2024-09-13T17:44:08.084135+0000","last_active":"2024-09-13T17:44:08.089264+0000","last_peered":"2024-09-13T17:44:08.089264+0000","last_clean":"2024-09-13T17:44:08.089264+0000","last_became_active":"2024-09-13T17:44:08.083885+0000","last_became_peered":"2024-09-13T17:44:08.083885+0000","last_unstale":"2024-09-13T17:44:08.089264+0000","last_undegraded":"2024-09-13T17:44:08.089264+0000","last_fullsized":"2024-09-13T17:44:08.089264+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:55.216920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.084497+0000","last_change":"2024-09-13T17:44:09.084497+0000","last_active":"2024-09-13T17:44:09.084497+0000","last_peered":"2024-09-13T17:44:09.084497+0000","last_clean":"2024-09-13T17:44:09.084497+0000","last_became_active":"2024-09-13T17:44:09.081961+0000","last_became_peered":"2024-09-13T17:44:09.081961+0000","last_unstale":"2024-09-13T17:44:09.084497+0000","last_undegraded":"2024-09-13T17:44:09.084497+0000","last_fullsized":"2024-09-13T17:44:09.084497+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:06:24.257175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.087438+0000","last_change":"2024-09-13T17:44:08.085352+0000","last_active":"2024-09-13T17:44:08.087438+0000","last_peered":"2024-09-13T17:44:08.087438+0000","last_clean":"2024-09-13T17:44:08.087438+0000","last_became_active":"2024-09-13T17:44:08.085113+0000","last_became_peered":"2024-09-13T17:44:08.085113+0000","last_unstale":"2024-09-13T17:44:08.087438+0000","last_undegraded":"2024-09-13T17:44:08.087438+0000","last_fullsized":"2024-09-13T17:44:08.087438+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:29:49.837938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.092521+0000","last_change":"2024-09-13T17:44:09.092521+0000","last_active":"2024-09-13T17:44:09.092521+0000","last_peered":"2024-09-13T17:44:09.092521+0000","last_clean":"2024-09-13T17:44:09.092521+0000","last_became_active":"2024-09-13T17:44:09.092212+0000","last_became_peered":"2024-09-13T17:44:09.092212+0000","last_unstale":"2024-09-13T17:44:09.092521+0000","last_undegraded":"2024-09-13T17:44:09.092521+0000","last_fullsized":"2024-09-13T17:44:09.092521+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:52.647827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.088970+0000","last_change":"2024-09-13T17:44:08.084591+0000","last_active":"2024-09-13T17:44:08.088970+0000","last_peered":"2024-09-13T17:44:08.088970+0000","last_clean":"2024-09-13T17:44:08.088970+0000","last_became_active":"2024-09-13T17:44:08.084345+0000","last_became_peered":"2024-09-13T17:44:08.084345+0000","last_unstale":"2024-09-13T17:44:08.088970+0000","last_undegraded":"2024-09-13T17:44:08.088970+0000","last_fullsized":"2024-09-13T17:44:08.088970+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:44:30.090059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":94,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.071200+0000","last_change":"2024-09-13T17:41:55.654539+0000","last_active":"2024-09-13T17:44:08.071200+0000","last_peered":"2024-09-13T17:44:08.071200+0000","last_clean":"2024-09-13T17:44:08.071200+0000","last_became_active":"2024-09-13T17:41:55.653536+0000","last_became_peered":"2024-09-13T17:41:55.653536+0000","last_unstale":"2024-09-13T17:44:08.071200+0000","last_undegraded":"2024-09-13T17:44:08.071200+0000","last_fullsized":"2024-09-13T17:44:08.071200+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:54.532965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:54.532965+0000","last_clean_scrub_stamp":"2024-09-13T17:41:54.532965+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:56.662588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":30,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.069418+0000","last_change":"2024-09-13T17:42:02.137356+0000","last_active":"2024-09-13T17:44:08.069418+0000","last_peered":"2024-09-13T17:44:08.069418+0000","last_clean":"2024-09-13T17:44:08.069418+0000","last_became_active":"2024-09-13T17:42:00.116534+0000","last_became_peered":"2024-09-13T17:42:00.116534+0000","last_unstale":"2024-09-13T17:44:08.069418+0000","last_undegraded":"2024-09-13T17:44:08.069418+0000","last_fullsized":"2024-09-13T17:44:08.069418+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:06.665067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000321799,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.076914+0000","last_change":"2024-09-13T17:44:09.076914+0000","last_active":"2024-09-13T17:44:09.076914+0000","last_peered":"2024-09-13T17:44:09.076914+0000","last_clean":"2024-09-13T17:44:09.076914+0000","last_became_active":"2024-09-13T17:44:09.076704+0000","last_became_peered":"2024-09-13T17:44:09.076704+0000","last_unstale":"2024-09-13T17:44:09.076914+0000","last_undegraded":"2024-09-13T17:44:09.076914+0000","last_fullsized":"2024-09-13T17:44:09.076914+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:00.260017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.099117+0000","last_change":"2024-09-13T17:44:08.081723+0000","last_active":"2024-09-13T17:44:09.099117+0000","last_peered":"2024-09-13T17:44:09.099117+0000","last_clean":"2024-09-13T17:44:09.099117+0000","last_became_active":"2024-09-13T17:44:08.081265+0000","last_became_peered":"2024-09-13T17:44:08.081265+0000","last_unstale":"2024-09-13T17:44:09.099117+0000","last_undegraded":"2024-09-13T17:44:09.099117+0000","last_fullsized":"2024-09-13T17:44:09.099117+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:28.105601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":29,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.067852+0000","last_change":"2024-09-13T17:42:02.137373+0000","last_active":"2024-09-13T17:44:08.067852+0000","last_peered":"2024-09-13T17:44:08.067852+0000","last_clean":"2024-09-13T17:44:08.067852+0000","last_became_active":"2024-09-13T17:42:00.116542+0000","last_became_peered":"2024-09-13T17:42:00.116542+0000","last_unstale":"2024-09-13T17:44:08.067852+0000","last_undegraded":"2024-09-13T17:44:08.067852+0000","last_fullsized":"2024-09-13T17:44:08.067852+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:53.754563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035063,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.080414+0000","last_change":"2024-09-13T17:44:09.080414+0000","last_active":"2024-09-13T17:44:09.080414+0000","last_peered":"2024-09-13T17:44:09.080414+0000","last_clean":"2024-09-13T17:44:09.080414+0000","last_became_active":"2024-09-13T17:44:09.080067+0000","last_became_peered":"2024-09-13T17:44:09.080067+0000","last_unstale":"2024-09-13T17:44:09.080414+0000","last_undegraded":"2024-09-13T17:44:09.080414+0000","last_fullsized":"2024-09-13T17:44:09.080414+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:46.797545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.091733+0000","last_change":"2024-09-13T17:44:08.089279+0000","last_active":"2024-09-13T17:44:08.091733+0000","last_peered":"2024-09-13T17:44:08.091733+0000","last_clean":"2024-09-13T17:44:08.091733+0000","last_became_active":"2024-09-13T17:44:08.089033+0000","last_became_peered":"2024-09-13T17:44:08.089033+0000","last_unstale":"2024-09-13T17:44:08.091733+0000","last_undegraded":"2024-09-13T17:44:08.091733+0000","last_fullsized":"2024-09-13T17:44:08.091733+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:48:54.288787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.066524+0000","last_change":"2024-09-13T17:42:02.615831+0000","last_active":"2024-09-13T17:44:08.066524+0000","last_peered":"2024-09-13T17:44:08.066524+0000","last_clean":"2024-09-13T17:44:08.066524+0000","last_became_active":"2024-09-13T17:42:00.116427+0000","last_became_peered":"2024-09-13T17:42:00.116427+0000","last_unstale":"2024-09-13T17:44:08.066524+0000","last_undegraded":"2024-09-13T17:44:08.066524+0000","last_fullsized":"2024-09-13T17:44:08.066524+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:29:17.848348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000228423,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.095690+0000","last_change":"2024-09-13T17:44:09.095690+0000","last_active":"2024-09-13T17:44:09.095690+0000","last_peered":"2024-09-13T17:44:09.095690+0000","last_clean":"2024-09-13T17:44:09.095690+0000","last_became_active":"2024-09-13T17:44:09.095448+0000","last_became_peered":"2024-09-13T17:44:09.095448+0000","last_unstale":"2024-09-13T17:44:09.095690+0000","last_undegraded":"2024-09-13T17:44:09.095690+0000","last_fullsized":"2024-09-13T17:44:09.095690+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:12:54.554173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.084187+0000","last_change":"2024-09-13T17:44:08.080099+0000","last_active":"2024-09-13T17:44:08.084187+0000","last_peered":"2024-09-13T17:44:08.084187+0000","last_clean":"2024-09-13T17:44:08.084187+0000","last_became_active":"2024-09-13T17:44:08.079856+0000","last_became_peered":"2024-09-13T17:44:08.079856+0000","last_unstale":"2024-09-13T17:44:08.084187+0000","last_undegraded":"2024-09-13T17:44:08.084187+0000","last_fullsized":"2024-09-13T17:44:08.084187+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:59.140826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":31,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.067921+0000","last_change":"2024-09-13T17:42:02.138081+0000","last_active":"2024-09-13T17:44:08.067921+0000","last_peered":"2024-09-13T17:44:08.067921+0000","last_clean":"2024-09-13T17:44:08.067921+0000","last_became_active":"2024-09-13T17:42:00.116465+0000","last_became_peered":"2024-09-13T17:42:00.116465+0000","last_unstale":"2024-09-13T17:44:08.067921+0000","last_undegraded":"2024-09-13T17:44:08.067921+0000","last_fullsized":"2024-09-13T17:44:08.067921+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:40:30.342531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001026573,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.085889+0000","last_change":"2024-09-13T17:44:09.085889+0000","last_active":"2024-09-13T17:44:09.085889+0000","last_peered":"2024-09-13T17:44:09.085889+0000","last_clean":"2024-09-13T17:44:09.085889+0000","last_became_active":"2024-09-13T17:44:09.085652+0000","last_became_peered":"2024-09-13T17:44:09.085652+0000","last_unstale":"2024-09-13T17:44:09.085889+0000","last_undegraded":"2024-09-13T17:44:09.085889+0000","last_fullsized":"2024-09-13T17:44:09.085889+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:44.944657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.091513+0000","last_change":"2024-09-13T17:44:08.086787+0000","last_active":"2024-09-13T17:44:08.091513+0000","last_peered":"2024-09-13T17:44:08.091513+0000","last_clean":"2024-09-13T17:44:08.091513+0000","last_became_active":"2024-09-13T17:44:08.086546+0000","last_became_peered":"2024-09-13T17:44:08.086546+0000","last_unstale":"2024-09-13T17:44:08.091513+0000","last_undegraded":"2024-09-13T17:44:08.091513+0000","last_fullsized":"2024-09-13T17:44:08.091513+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:31.939247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.073003+0000","last_change":"2024-09-13T17:42:02.135135+0000","last_active":"2024-09-13T17:44:08.073003+0000","last_peered":"2024-09-13T17:44:08.073003+0000","last_clean":"2024-09-13T17:44:08.073003+0000","last_became_active":"2024-09-13T17:42:00.364240+0000","last_became_peered":"2024-09-13T17:42:00.364240+0000","last_unstale":"2024-09-13T17:44:08.073003+0000","last_undegraded":"2024-09-13T17:44:08.073003+0000","last_fullsized":"2024-09-13T17:44:08.073003+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:31:29.382006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575944,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.068199+0000","last_change":"2024-09-13T17:44:09.068199+0000","last_active":"2024-09-13T17:44:09.068199+0000","last_peered":"2024-09-13T17:44:09.068199+0000","last_clean":"2024-09-13T17:44:09.068199+0000","last_became_active":"2024-09-13T17:44:09.067939+0000","last_became_peered":"2024-09-13T17:44:09.067939+0000","last_unstale":"2024-09-13T17:44:09.068199+0000","last_undegraded":"2024-09-13T17:44:09.068199+0000","last_fullsized":"2024-09-13T17:44:09.068199+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:08:00.966936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.091636+0000","last_change":"2024-09-13T17:44:08.088268+0000","last_active":"2024-09-13T17:44:08.091636+0000","last_peered":"2024-09-13T17:44:08.091636+0000","last_clean":"2024-09-13T17:44:08.091636+0000","last_became_active":"2024-09-13T17:44:08.087998+0000","last_became_peered":"2024-09-13T17:44:08.087998+0000","last_unstale":"2024-09-13T17:44:08.091636+0000","last_undegraded":"2024-09-13T17:44:08.091636+0000","last_fullsized":"2024-09-13T17:44:08.091636+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:55.216920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":29,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.071260+0000","last_change":"2024-09-13T17:42:02.138136+0000","last_active":"2024-09-13T17:44:08.071260+0000","last_peered":"2024-09-13T17:44:08.071260+0000","last_clean":"2024-09-13T17:44:08.071260+0000","last_became_active":"2024-09-13T17:42:00.364233+0000","last_became_peered":"2024-09-13T17:42:00.364233+0000","last_unstale":"2024-09-13T17:44:08.071260+0000","last_undegraded":"2024-09-13T17:44:08.071260+0000","last_fullsized":"2024-09-13T17:44:08.071260+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:54.234194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010757130000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.092485+0000","last_change":"2024-09-13T17:44:09.092485+0000","last_active":"2024-09-13T17:44:09.092485+0000","last_peered":"2024-09-13T17:44:09.092485+0000","last_clean":"2024-09-13T17:44:09.092485+0000","last_became_active":"2024-09-13T17:44:09.092232+0000","last_became_peered":"2024-09-13T17:44:09.092232+0000","last_unstale":"2024-09-13T17:44:09.092485+0000","last_undegraded":"2024-09-13T17:44:09.092485+0000","last_fullsized":"2024-09-13T17:44:09.092485+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:47.365057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.094767+0000","last_change":"2024-09-13T17:44:08.091205+0000","last_active":"2024-09-13T17:44:08.094767+0000","last_peered":"2024-09-13T17:44:08.094767+0000","last_clean":"2024-09-13T17:44:08.094767+0000","last_became_active":"2024-09-13T17:44:08.090931+0000","last_became_peered":"2024-09-13T17:44:08.090931+0000","last_unstale":"2024-09-13T17:44:08.094767+0000","last_undegraded":"2024-09-13T17:44:08.094767+0000","last_fullsized":"2024-09-13T17:44:08.094767+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:43.940369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":29,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.070560+0000","last_change":"2024-09-13T17:42:02.134963+0000","last_active":"2024-09-13T17:44:08.070560+0000","last_peered":"2024-09-13T17:44:08.070560+0000","last_clean":"2024-09-13T17:44:08.070560+0000","last_became_active":"2024-09-13T17:42:00.364233+0000","last_became_peered":"2024-09-13T17:42:00.364233+0000","last_unstale":"2024-09-13T17:44:08.070560+0000","last_undegraded":"2024-09-13T17:44:08.070560+0000","last_fullsized":"2024-09-13T17:44:08.070560+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:24:31.191067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393223,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T17:44:09.095839+0000","last_change":"2024-09-13T17:44:09.095839+0000","last_active":"2024-09-13T17:44:09.095839+0000","last_peered":"2024-09-13T17:44:09.095839+0000","last_clean":"2024-09-13T17:44:09.095839+0000","last_became_active":"2024-09-13T17:44:09.095323+0000","last_became_peered":"2024-09-13T17:44:09.095323+0000","last_unstale":"2024-09-13T17:44:09.095839+0000","last_undegraded":"2024-09-13T17:44:09.095839+0000","last_fullsized":"2024-09-13T17:44:09.095839+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:08.052539+0000","last_clean_scrub_stamp":"2024-09-13T17:44:08.052539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:23:36.075940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.082770+0000","last_change":"2024-09-13T17:44:08.080518+0000","last_active":"2024-09-13T17:44:08.082770+0000","last_peered":"2024-09-13T17:44:08.082770+0000","last_clean":"2024-09-13T17:44:08.082770+0000","last_became_active":"2024-09-13T17:44:08.080255+0000","last_became_peered":"2024-09-13T17:44:08.080255+0000","last_unstale":"2024-09-13T17:44:08.082770+0000","last_undegraded":"2024-09-13T17:44:08.082770+0000","last_fullsized":"2024-09-13T17:44:08.082770+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:07.048251+0000","last_clean_scrub_stamp":"2024-09-13T17:44:07.048251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:23:35.071651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":29,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T17:44:08.071822+0000","last_change":"2024-09-13T17:42:02.138362+0000","last_active":"2024-09-13T17:44:08.071822+0000","last_peered":"2024-09-13T17:44:08.071822+0000","last_clean":"2024-09-13T17:44:08.071822+0000","last_became_active":"2024-09-13T17:42:00.364214+0000","last_became_peered":"2024-09-13T17:42:00.364214+0000","last_unstale":"2024-09-13T17:44:08.071822+0000","last_undegraded":"2024-09-13T17:44:08.071822+0000","last_fullsized":"2024-09-13T17:44:08.071822+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:33.976642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012776910000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705694,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297620,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074220,"statfs":{"total":96636764160,"available":96332001280,"internally_reserved":0,"allocated":266240,"data_stored":93848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:44:00 2024","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.30399999999999999,"15min":0.30499999999999999},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.39800000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.29299999999999998,"5min":0.317,"15min":0.32100000000000001},"min":{"1min":0.223,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.45300000000000001,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.35399999999999998}]},{"osd":1,"last update":"Fri Sep 13 17:44:00 2024","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.34899999999999998,"15min":0.35299999999999998},"min":{"1min":0.255,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.5,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.30599999999999999,"5min":0.318,"15min":0.32000000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.55200000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.33800000000000002}]},{"osd":2,"last update":"Fri Sep 13 17:44:00 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.36899999999999999,"15min":0.372},"min":{"1min":0.28000000000000003,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.51100000000000001,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.33700000000000002,"15min":0.33800000000000002},"min":{"1min":0.23499999999999999,"5min":0.216,"15min":0.216},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.40699999999999997}]}]},{"osd":2,"up_from":9,"seq":38654705694,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297616,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074224,"statfs":{"total":96636764160,"available":96332005376,"internally_reserved":0,"allocated":262144,"data_stored":93829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:43:59 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.31,"15min":0.307},"min":{"1min":0.22500000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.34599999999999997,"15min":0.34999999999999998},"min":{"1min":0.22700000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.30099999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:43:59 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35199999999999998,"15min":0.35299999999999998},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.252},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.35699999999999998,"15min":0.35799999999999998},"min":{"1min":0.24399999999999999,"5min":0.224,"15min":0.224},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.32100000000000001}]},{"osd":3,"last update":"Fri Sep 13 17:43:59 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.377,"15min":0.379},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.376,"5min":0.372,"15min":0.371},"min":{"1min":0.28599999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.28699999999999998}]}]},{"osd":1,"up_from":9,"seq":38654705697,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298216,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073624,"statfs":{"total":96636764160,"available":96331390976,"internally_reserved":0,"allocated":880640,"data_stored":689584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:43:56 2024","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.40300000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.31,"15min":0.307},"min":{"1min":0.25,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"last":0.27700000000000002}]},{"osd":2,"last update":"Fri Sep 13 17:43:56 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.318,"15min":0.317},"min":{"1min":0.23200000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"last":0.248},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.251,"5min":0.222,"15min":0.222},"max":{"1min":0.44800000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"last":0.30299999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:43:56 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.34799999999999998,"15min":0.34999999999999998},"min":{"1min":0.246,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"last":0.312},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35999999999999999,"15min":0.36099999999999999},"min":{"1min":0.26800000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.46500000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.29499999999999998}]}]},{"osd":0,"up_from":9,"seq":38654705695,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298196,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073644,"statfs":{"total":96636764160,"available":96331411456,"internally_reserved":0,"allocated":856064,"data_stored":684197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:43:57 2024","interfaces":[{"interface":"back","average":{"1min":0.26700000000000002,"5min":0.29999999999999999,"15min":0.30599999999999999},"min":{"1min":0.214,"5min":0.19,"15min":0.19},"max":{"1min":0.38200000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.307,"5min":0.311,"15min":0.311},"min":{"1min":0.22900000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.42299999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"last":0.28999999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:43:57 2024","interfaces":[{"interface":"back","average":{"1min":0.29799999999999999,"5min":0.318,"15min":0.32100000000000001},"min":{"1min":0.23499999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.41599999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.31,"5min":0.32400000000000001,"15min":0.32600000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.437,"5min":0.44600000000000001,"15min":0.44600000000000001},"last":0.249}]},{"osd":3,"last update":"Fri Sep 13 17:43:57 2024","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.33200000000000002,"15min":0.33300000000000002},"min":{"1min":0.249,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.39700000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.316,"5min":0.33200000000000002,"15min":0.33500000000000002},"min":{"1min":0.254,"5min":0.248,"15min":0.248},"max":{"1min":0.40999999999999998,"5min":0.504,"15min":0.504},"last":0.23499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:44:12.267 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-09-13T17:44:12.267 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-13T17:44:12.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.401+0000 7fd21bb5d640 1 Processor -- start 2024-09-13T17:44:12.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.401+0000 7fd21bb5d640 1 -- start start 2024-09-13T17:44:12.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.401+0000 7fd21bb5d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd214097300 0x7fd2140976d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:12.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.401+0000 7fd21bb5d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd214097c10 0x7fd21412fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:12.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.401+0000 7fd21bb5d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd2141303e0 0x7fd2141327c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:12.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.401+0000 7fd21bb5d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd21409a660 con 0x7fd214097300 2024-09-13T17:44:12.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.401+0000 7fd21bb5d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd21406ebd0 con 0x7fd214097c10 2024-09-13T17:44:12.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.401+0000 7fd21bb5d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd214098df0 con 0x7fd2141303e0 2024-09-13T17:44:12.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.402+0000 7fd2198f9640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd214097300 0x7fd2140976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:12.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.402+0000 7fd2190f8640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd214097c10 0x7fd21412fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:12.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.402+0000 7fd21a0fa640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd2141303e0 0x7fd2141327c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:12.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.402+0000 7fd2198f9640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd214097300 0x7fd2140976d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56896/0 (socket says 172.21.15.102:56896) 2024-09-13T17:44:12.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.402+0000 7fd2190f8640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd214097c10 0x7fd21412fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35152/0 (socket says 172.21.15.102:35152) 2024-09-13T17:44:12.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.402+0000 7fd2198f9640 1 -- 172.21.15.102:0/1066191009 learned_addr learned my addr 172.21.15.102:0/1066191009 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:12.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.402+0000 7fd2190f8640 1 -- 172.21.15.102:0/1066191009 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd2141303e0 msgr2=0x7fd2141327c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:12.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.402+0000 7fd2190f8640 1 --2- 172.21.15.102:0/1066191009 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd2141303e0 0x7fd2141327c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.402+0000 7fd2190f8640 1 -- 172.21.15.102:0/1066191009 >> v2:172.21.15.102:3300/0 conn(0x7fd214097300 msgr2=0x7fd2140976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:12.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.402+0000 7fd2190f8640 1 --2- 172.21.15.102:0/1066191009 >> v2:172.21.15.102:3300/0 conn(0x7fd214097300 0x7fd2140976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.402+0000 7fd2190f8640 1 -- 172.21.15.102:0/1066191009 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd214099960 con 0x7fd214097c10 2024-09-13T17:44:12.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.402+0000 7fd2198f9640 1 --2- 172.21.15.102:0/1066191009 >> v2:172.21.15.102:3300/0 conn(0x7fd214097300 0x7fd2140976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:12.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.402+0000 7fd21a0fa640 1 --2- 172.21.15.102:0/1066191009 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd2141303e0 0x7fd2141327c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:12.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.403+0000 7fd2190f8640 1 --2- 172.21.15.102:0/1066191009 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd214097c10 0x7fd21412fea0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd204009850 tx=0x7fd2040090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fe6cbea531053a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:12.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.403+0000 7fd2188f7640 1 -- 172.21.15.102:0/1066191009 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20402ac50 con 0x7fd214097c10 2024-09-13T17:44:12.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.403+0000 7fd2188f7640 1 -- 172.21.15.102:0/1066191009 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd20402adb0 con 0x7fd214097c10 2024-09-13T17:44:12.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.403+0000 7fd2188f7640 1 -- 172.21.15.102:0/1066191009 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204032670 con 0x7fd214097c10 2024-09-13T17:44:12.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.403+0000 7fd21bb5d640 1 -- 172.21.15.102:0/1066191009 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd214097c10 msgr2=0x7fd21412fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:12.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.403+0000 7fd21bb5d640 1 --2- 172.21.15.102:0/1066191009 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd214097c10 0x7fd21412fea0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd204009850 tx=0x7fd2040090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.403+0000 7fd21bb5d640 1 -- 172.21.15.102:0/1066191009 shutdown_connections 2024-09-13T17:44:12.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.403+0000 7fd21bb5d640 1 --2- 172.21.15.102:0/1066191009 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd2141303e0 0x7fd2141327c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.403+0000 7fd21bb5d640 1 --2- 172.21.15.102:0/1066191009 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd214097c10 0x7fd21412fea0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.403+0000 7fd21bb5d640 1 --2- 172.21.15.102:0/1066191009 >> v2:172.21.15.102:3300/0 conn(0x7fd214097300 0x7fd2140976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.403+0000 7fd21bb5d640 1 -- 172.21.15.102:0/1066191009 >> 172.21.15.102:0/1066191009 conn(0x7fd21409e6b0 msgr2=0x7fd21409eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:12.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.403+0000 7fd21bb5d640 1 -- 172.21.15.102:0/1066191009 shutdown_connections 2024-09-13T17:44:12.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.403+0000 7fd21bb5d640 1 -- 172.21.15.102:0/1066191009 wait complete. 2024-09-13T17:44:12.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.404+0000 7fd21bb5d640 1 Processor -- start 2024-09-13T17:44:12.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.404+0000 7fd21bb5d640 1 -- start start 2024-09-13T17:44:12.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.405+0000 7fd21bb5d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd214097300 0x7fd21411dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:12.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.405+0000 7fd21bb5d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd214097c10 0x7fd21411e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:12.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.405+0000 7fd21bb5d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd2141303e0 0x7fd21413eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:12.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.405+0000 7fd21bb5d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21411da90 con 0x7fd214097c10 2024-09-13T17:44:12.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.405+0000 7fd21bb5d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd21409df60 con 0x7fd214097300 2024-09-13T17:44:12.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.405+0000 7fd21bb5d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd21406ddf0 con 0x7fd2141303e0 2024-09-13T17:44:12.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.405+0000 7fd2198f9640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd214097300 0x7fd21411dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:12.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.405+0000 7fd2190f8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd214097c10 0x7fd21411e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:12.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.405+0000 7fd2198f9640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd214097300 0x7fd21411dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35166/0 (socket says 172.21.15.102:35166) 2024-09-13T17:44:12.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.405+0000 7fd2198f9640 1 -- 172.21.15.102:0/737146617 learned_addr learned my addr 172.21.15.102:0/737146617 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:12.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.405+0000 7fd21a0fa640 1 --2- 172.21.15.102:0/737146617 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd2141303e0 0x7fd21413eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:12.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.405+0000 7fd2198f9640 1 -- 172.21.15.102:0/737146617 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd2141303e0 msgr2=0x7fd21413eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:12.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.405+0000 7fd2198f9640 1 --2- 172.21.15.102:0/737146617 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd2141303e0 0x7fd21413eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.405+0000 7fd2198f9640 1 -- 172.21.15.102:0/737146617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd214097c10 msgr2=0x7fd21411e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:12.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.405+0000 7fd2198f9640 1 --2- 172.21.15.102:0/737146617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd214097c10 0x7fd21411e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.405+0000 7fd2198f9640 1 -- 172.21.15.102:0/737146617 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd210009450 con 0x7fd214097300 2024-09-13T17:44:12.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.405+0000 7fd2190f8640 1 --2- 172.21.15.102:0/737146617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd214097c10 0x7fd21411e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:12.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.405+0000 7fd2198f9640 1 --2- 172.21.15.102:0/737146617 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd214097300 0x7fd21411dcc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd21000ec80 tx=0x7fd2100091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:12.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.405+0000 7fd209ffb640 1 -- 172.21.15.102:0/737146617 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21000eec0 con 0x7fd214097300 2024-09-13T17:44:12.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.405+0000 7fd209ffb640 1 -- 172.21.15.102:0/737146617 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd21000cbc0 con 0x7fd214097300 2024-09-13T17:44:12.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.405+0000 7fd21bb5d640 1 -- 172.21.15.102:0/737146617 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd20c00bc30 con 0x7fd214097300 2024-09-13T17:44:12.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.405+0000 7fd209ffb640 1 -- 172.21.15.102:0/737146617 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd210013470 con 0x7fd214097300 2024-09-13T17:44:12.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.406+0000 7fd21bb5d640 1 -- 172.21.15.102:0/737146617 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd204009390 con 0x7fd214097300 2024-09-13T17:44:12.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.406+0000 7fd209ffb640 1 -- 172.21.15.102:0/737146617 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fd210013820 con 0x7fd214097300 2024-09-13T17:44:12.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.406+0000 7fd209ffb640 1 -- 172.21.15.102:0/737146617 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3660+0+0 (secure 0 0 0) 0x7fd210042430 con 0x7fd214097300 2024-09-13T17:44:12.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.407+0000 7fd21bb5d640 1 -- 172.21.15.102:0/737146617 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd214120e20 con 0x7fd214097300 2024-09-13T17:44:12.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.411+0000 7fd209ffb640 1 -- 172.21.15.102:0/737146617 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fd210021080 con 0x7fd214097300 2024-09-13T17:44:12.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.615+0000 7fd21bb5d640 1 -- 172.21.15.102:0/737146617 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fd214064000 con 0x7fd214097300 2024-09-13T17:44:12.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.621+0000 7fd209ffb640 1 -- 172.21.15.102:0/737146617 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 190+0+0 (secure 0 0 0) 0x7fd2100236b0 con 0x7fd214097300 2024-09-13T17:44:12.623 INFO:teuthology.orchestra.run.smithi102.stderr:updated caps for client.0 2024-09-13T17:44:12.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.623+0000 7fd21bb5d640 1 -- 172.21.15.102:0/737146617 >> v1:172.21.15.102:6816/34937 conn(0x7fd1f8031190 legacy=0x7fd1f8033640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:12.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.623+0000 7fd21bb5d640 1 -- 172.21.15.102:0/737146617 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd214097300 msgr2=0x7fd21411dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:12.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.623+0000 7fd21bb5d640 1 --2- 172.21.15.102:0/737146617 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd214097300 0x7fd21411dcc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd21000ec80 tx=0x7fd2100091d0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.623+0000 7fd21bb5d640 1 -- 172.21.15.102:0/737146617 shutdown_connections 2024-09-13T17:44:12.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.623+0000 7fd21bb5d640 1 --2- 172.21.15.102:0/737146617 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd2141303e0 0x7fd21413eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.623+0000 7fd21bb5d640 1 --2- 172.21.15.102:0/737146617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd214097c10 0x7fd21411e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.623+0000 7fd21bb5d640 1 --2- 172.21.15.102:0/737146617 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd214097300 0x7fd21411dcc0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.623+0000 7fd21bb5d640 1 -- 172.21.15.102:0/737146617 >> 172.21.15.102:0/737146617 conn(0x7fd21409e6b0 msgr2=0x7fd21409c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:12.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.623+0000 7fd21bb5d640 1 -- 172.21.15.102:0/737146617 shutdown_connections 2024-09-13T17:44:12.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.623+0000 7fd21bb5d640 1 -- 172.21.15.102:0/737146617 wait complete. 2024-09-13T17:44:12.635 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:44:12.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.727+0000 7fc9e8ec0640 1 Processor -- start 2024-09-13T17:44:12.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.727+0000 7fc9e8ec0640 1 -- start start 2024-09-13T17:44:12.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.727+0000 7fc9e8ec0640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc9e406b140 0x7fc9e40998c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:12.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.727+0000 7fc9e8ec0640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc9e409ac20 0x7fc9e4099e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:12.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.727+0000 7fc9e8ec0640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc9e409a340 0x7fc9e412d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:12.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.727+0000 7fc9e8ec0640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9e406a680 con 0x7fc9e409a340 2024-09-13T17:44:12.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.727+0000 7fc9e8ec0640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e406ee90 con 0x7fc9e409ac20 2024-09-13T17:44:12.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.727+0000 7fc9e8ec0640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e406ac90 con 0x7fc9e406b140 2024-09-13T17:44:12.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.728+0000 7fc9e1d9b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc9e409ac20 0x7fc9e4099e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:12.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.728+0000 7fc9e259c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc9e406b140 0x7fc9e40998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:12.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.728+0000 7fc9e2d9d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc9e409a340 0x7fc9e412d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:12.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.728+0000 7fc9e1d9b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc9e409ac20 0x7fc9e4099e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35180/0 (socket says 172.21.15.102:35180) 2024-09-13T17:44:12.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.728+0000 7fc9e2d9d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fc9e409a340 0x7fc9e412d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56914/0 (socket says 172.21.15.102:56914) 2024-09-13T17:44:12.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.728+0000 7fc9e259c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc9e406b140 0x7fc9e40998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:57704/0 (socket says 172.21.15.102:57704) 2024-09-13T17:44:12.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.728+0000 7fc9e1d9b640 1 -- 172.21.15.102:0/411003238 learned_addr learned my addr 172.21.15.102:0/411003238 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:12.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.728+0000 7fc9e2d9d640 1 -- 172.21.15.102:0/411003238 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc9e406b140 msgr2=0x7fc9e40998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:12.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.728+0000 7fc9e2d9d640 1 --2- 172.21.15.102:0/411003238 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc9e406b140 0x7fc9e40998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.728+0000 7fc9e2d9d640 1 -- 172.21.15.102:0/411003238 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc9e409ac20 msgr2=0x7fc9e4099e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:12.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.728+0000 7fc9e2d9d640 1 --2- 172.21.15.102:0/411003238 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc9e409ac20 0x7fc9e4099e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.728+0000 7fc9e2d9d640 1 -- 172.21.15.102:0/411003238 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e4099510 con 0x7fc9e409a340 2024-09-13T17:44:12.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.728+0000 7fc9e1d9b640 1 --2- 172.21.15.102:0/411003238 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc9e409ac20 0x7fc9e4099e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:12.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.728+0000 7fc9e259c640 1 --2- 172.21.15.102:0/411003238 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc9e406b140 0x7fc9e40998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:12.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.729+0000 7fc9e2d9d640 1 --2- 172.21.15.102:0/411003238 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9e409a340 0x7fc9e412d280 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc009e60 tx=0x7fc9dc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3a5c52aa1d3796e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:12.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.729+0000 7fc9e159a640 1 -- 172.21.15.102:0/411003238 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc007550 con 0x7fc9e409a340 2024-09-13T17:44:12.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.729+0000 7fc9e159a640 1 -- 172.21.15.102:0/411003238 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9dc02d050 con 0x7fc9e409a340 2024-09-13T17:44:12.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.729+0000 7fc9e159a640 1 -- 172.21.15.102:0/411003238 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc007920 con 0x7fc9e409a340 2024-09-13T17:44:12.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.729+0000 7fc9e8ec0640 1 -- 172.21.15.102:0/411003238 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9e409a340 msgr2=0x7fc9e412d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:12.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.729+0000 7fc9e8ec0640 1 --2- 172.21.15.102:0/411003238 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9e409a340 0x7fc9e412d280 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc009e60 tx=0x7fc9dc009780 comp rx=0 tx=0).stop 2024-09-13T17:44:12.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.729+0000 7fc9e8ec0640 1 -- 172.21.15.102:0/411003238 shutdown_connections 2024-09-13T17:44:12.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.729+0000 7fc9e8ec0640 1 --2- 172.21.15.102:0/411003238 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9e409a340 0x7fc9e412d280 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.729+0000 7fc9e8ec0640 1 --2- 172.21.15.102:0/411003238 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc9e409ac20 0x7fc9e4099e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.729+0000 7fc9e8ec0640 1 --2- 172.21.15.102:0/411003238 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc9e406b140 0x7fc9e40998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.729+0000 7fc9e8ec0640 1 -- 172.21.15.102:0/411003238 >> 172.21.15.102:0/411003238 conn(0x7fc9e409e820 msgr2=0x7fc9e409ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:12.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.729+0000 7fc9e8ec0640 1 -- 172.21.15.102:0/411003238 shutdown_connections 2024-09-13T17:44:12.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.729+0000 7fc9e8ec0640 1 -- 172.21.15.102:0/411003238 wait complete. 2024-09-13T17:44:12.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.730+0000 7fc9e8ec0640 1 Processor -- start 2024-09-13T17:44:12.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.730+0000 7fc9e8ec0640 1 -- start start 2024-09-13T17:44:12.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.731+0000 7fc9e8ec0640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc9e406b140 0x7fc9e411f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:12.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.731+0000 7fc9e8ec0640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9e409a340 0x7fc9e411bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:12.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.731+0000 7fc9e8ec0640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc9e409ac20 0x7fc9e411c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:12.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.731+0000 7fc9e8ec0640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e411d1e0 con 0x7fc9e409a340 2024-09-13T17:44:12.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.731+0000 7fc9e8ec0640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e409e0d0 con 0x7fc9e409ac20 2024-09-13T17:44:12.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.731+0000 7fc9e8ec0640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e4069dd0 con 0x7fc9e406b140 2024-09-13T17:44:12.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.731+0000 7fc9e259c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc9e406b140 0x7fc9e411f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:12.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.731+0000 7fc9e1d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9e409a340 0x7fc9e411bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:12.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.731+0000 7fc9e259c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc9e406b140 0x7fc9e411f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:57710/0 (socket says 172.21.15.102:57710) 2024-09-13T17:44:12.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.731+0000 7fc9e259c640 1 -- 172.21.15.102:0/679864560 learned_addr learned my addr 172.21.15.102:0/679864560 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:12.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.731+0000 7fc9e1d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9e409a340 0x7fc9e411bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56918/0 (socket says 172.21.15.102:56918) 2024-09-13T17:44:12.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.731+0000 7fc9e2d9d640 1 --2- 172.21.15.102:0/679864560 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc9e409ac20 0x7fc9e411c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:12.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.731+0000 7fc9e1d9b640 1 -- 172.21.15.102:0/679864560 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc9e406b140 msgr2=0x7fc9e411f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:12.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.731+0000 7fc9e1d9b640 1 --2- 172.21.15.102:0/679864560 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc9e406b140 0x7fc9e411f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.731+0000 7fc9e1d9b640 1 -- 172.21.15.102:0/679864560 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc9e409ac20 msgr2=0x7fc9e411c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:12.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.731+0000 7fc9e1d9b640 1 --2- 172.21.15.102:0/679864560 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc9e409ac20 0x7fc9e411c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.731+0000 7fc9e1d9b640 1 -- 172.21.15.102:0/679864560 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d4009390 con 0x7fc9e409a340 2024-09-13T17:44:12.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.731+0000 7fc9e259c640 1 --2- 172.21.15.102:0/679864560 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc9e406b140 0x7fc9e411f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:12.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.731+0000 7fc9e2d9d640 1 --2- 172.21.15.102:0/679864560 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc9e409ac20 0x7fc9e411c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:12.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.731+0000 7fc9e1d9b640 1 --2- 172.21.15.102:0/679864560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9e409a340 0x7fc9e411bc10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8009fd0 tx=0x7fc9d80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:12.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.731+0000 7fc9d27fc640 1 -- 172.21.15.102:0/679864560 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d8009680 con 0x7fc9e409a340 2024-09-13T17:44:12.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.731+0000 7fc9d27fc640 1 -- 172.21.15.102:0/679864560 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d800c940 con 0x7fc9e409a340 2024-09-13T17:44:12.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.731+0000 7fc9e8ec0640 1 -- 172.21.15.102:0/679864560 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9d8009450 con 0x7fc9e409a340 2024-09-13T17:44:12.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.731+0000 7fc9d27fc640 1 -- 172.21.15.102:0/679864560 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d800cce0 con 0x7fc9e409a340 2024-09-13T17:44:12.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.731+0000 7fc9e8ec0640 1 -- 172.21.15.102:0/679864560 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9dc009a20 con 0x7fc9e409a340 2024-09-13T17:44:12.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.732+0000 7fc9d27fc640 1 -- 172.21.15.102:0/679864560 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fc9d8013680 con 0x7fc9e409a340 2024-09-13T17:44:12.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.732+0000 7fc9d27fc640 1 -- 172.21.15.102:0/679864560 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3660+0+0 (secure 0 0 0) 0x7fc9d8042320 con 0x7fc9e409a340 2024-09-13T17:44:12.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.732+0000 7fc9e8ec0640 1 -- 172.21.15.102:0/679864560 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e4064000 con 0x7fc9e409a340 2024-09-13T17:44:12.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.737+0000 7fc9d27fc640 1 -- 172.21.15.102:0/679864560 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fc9d8021080 con 0x7fc9e409a340 2024-09-13T17:44:12.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.918+0000 7fc9e8ec0640 1 -- 172.21.15.102:0/679864560 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9e4067690 con 0x7fc9e409a340 2024-09-13T17:44:12.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.919+0000 7fc9d27fc640 1 -- 172.21.15.102:0/679864560 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2268 (secure 0 0 0) 0x7fc9d80236b0 con 0x7fc9e409a340 2024-09-13T17:44:12.921 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:44:12.921 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:44:08.385597+0000","modified":"2024-09-13T17:44:11.073569+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5078},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5078":{"gid":5078,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6817/2209249660","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":2209249660}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-09-13T17:44:12.921 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 11 2024-09-13T17:44:12.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.920+0000 7fc9e8ec0640 1 -- 172.21.15.102:0/679864560 >> v1:172.21.15.102:6816/34937 conn(0x7fc9c4031160 legacy=0x7fc9c4033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:12.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.920+0000 7fc9e8ec0640 1 -- 172.21.15.102:0/679864560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9e409a340 msgr2=0x7fc9e411bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:12.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.920+0000 7fc9e8ec0640 1 --2- 172.21.15.102:0/679864560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9e409a340 0x7fc9e411bc10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8009fd0 tx=0x7fc9d80091d0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.920+0000 7fc9e8ec0640 1 -- 172.21.15.102:0/679864560 shutdown_connections 2024-09-13T17:44:12.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.920+0000 7fc9e8ec0640 1 --2- 172.21.15.102:0/679864560 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc9e409ac20 0x7fc9e411c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.920+0000 7fc9e8ec0640 1 --2- 172.21.15.102:0/679864560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9e409a340 0x7fc9e411bc10 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.920+0000 7fc9e8ec0640 1 --2- 172.21.15.102:0/679864560 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc9e406b140 0x7fc9e411f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.920+0000 7fc9e8ec0640 1 -- 172.21.15.102:0/679864560 >> 172.21.15.102:0/679864560 conn(0x7fc9e409e820 msgr2=0x7fc9e406cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:12.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.921+0000 7fc9e8ec0640 1 -- 172.21.15.102:0/679864560 shutdown_connections 2024-09-13T17:44:12.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:12.921+0000 7fc9e8ec0640 1 -- 172.21.15.102:0/679864560 wait complete. 2024-09-13T17:44:13.033 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 11, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T17:44:08.385597+0000', 'modified': '2024-09-13T17:44:11.073569+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5078}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5078': {'gid': 5078, 'name': 'a', 'rank': 0, 'incarnation': 9, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.102:6817/2209249660', 'addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.102:6817', 'nonce': 2209249660}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-09-13T17:44:13.034 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T17:44:13.034 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-13T17:44:13.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.128+0000 7f2cd279d640 1 Processor -- start 2024-09-13T17:44:13.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.128+0000 7f2cd279d640 1 -- start start 2024-09-13T17:44:13.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.128+0000 7f2cd279d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2ccc09abb0 0x7f2ccc12c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.128+0000 7f2cd279d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ccc12cae0 0x7f2ccc097490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.128+0000 7f2cd279d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ccc06a530 con 0x7f2ccc12c6a0 2024-09-13T17:44:13.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.128+0000 7f2cd279d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ccc09af80 con 0x7f2ccc09abb0 2024-09-13T17:44:13.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.128+0000 7f2cd279d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ccc098be0 con 0x7f2ccc12cae0 2024-09-13T17:44:13.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.128+0000 7f2ccbfff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2ccc09abb0 0x7f2ccc12c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.128+0000 7f2cd0d3a640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ccc12cae0 0x7f2ccc097490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.128+0000 7f2ccb7fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f2ccc12c6a0 0x7f2ccc096d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57204/0 (socket says 172.21.15.102:57204) 2024-09-13T17:44:13.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.128+0000 7f2ccb7fe640 1 -- 172.21.15.102:0/2861881339 learned_addr learned my addr 172.21.15.102:0/2861881339 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:44:13.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.129+0000 7f2cd0d3a640 1 -- 172.21.15.102:0/2861881339 >> v2:172.21.15.102:3300/0 conn(0x7f2ccc09abb0 msgr2=0x7f2ccc12c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:13.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.129+0000 7f2cd0d3a640 1 --2- 172.21.15.102:0/2861881339 >> v2:172.21.15.102:3300/0 conn(0x7f2ccc09abb0 0x7f2ccc12c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.129+0000 7f2cd0d3a640 1 -- 172.21.15.102:0/2861881339 >> v1:172.21.15.102:6789/0 conn(0x7f2ccc12c6a0 legacy=0x7f2ccc096d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:13.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.129+0000 7f2ccaffd640 1 -- 172.21.15.102:0/2861881339 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4166096416 0 0) 0x7f2ccc06a530 con 0x7f2ccc12c6a0 2024-09-13T17:44:13.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.129+0000 7f2cd0d3a640 1 -- 172.21.15.102:0/2861881339 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ccc098930 con 0x7f2ccc12cae0 2024-09-13T17:44:13.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.129+0000 7f2ccbfff640 1 --2- 172.21.15.102:0/2861881339 >> v2:172.21.15.102:3300/0 conn(0x7f2ccc09abb0 0x7f2ccc12c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:13.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.129+0000 7f2cd0d3a640 1 --2- 172.21.15.102:0/2861881339 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ccc12cae0 0x7f2ccc097490 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4009e60 tx=0x7f2cc4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af1b42db911a66c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:13.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.129+0000 7f2ccaffd640 1 -- 172.21.15.102:0/2861881339 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4007550 con 0x7f2ccc12cae0 2024-09-13T17:44:13.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.129+0000 7f2ccaffd640 1 -- 172.21.15.102:0/2861881339 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc402d050 con 0x7f2ccc12cae0 2024-09-13T17:44:13.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.129+0000 7f2ccaffd640 1 -- 172.21.15.102:0/2861881339 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc40078e0 con 0x7f2ccc12cae0 2024-09-13T17:44:13.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.130+0000 7f2cd279d640 1 -- 172.21.15.102:0/2861881339 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ccc12cae0 msgr2=0x7f2ccc097490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:13.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.130+0000 7f2cd279d640 1 --2- 172.21.15.102:0/2861881339 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ccc12cae0 0x7f2ccc097490 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4009e60 tx=0x7f2cc4009780 comp rx=0 tx=0).stop 2024-09-13T17:44:13.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.130+0000 7f2cd279d640 1 -- 172.21.15.102:0/2861881339 shutdown_connections 2024-09-13T17:44:13.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.130+0000 7f2cd279d640 1 --2- 172.21.15.102:0/2861881339 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ccc12cae0 0x7f2ccc097490 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.130+0000 7f2cd279d640 1 --2- 172.21.15.102:0/2861881339 >> v2:172.21.15.102:3300/0 conn(0x7f2ccc09abb0 0x7f2ccc12c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.130+0000 7f2cd279d640 1 -- 172.21.15.102:0/2861881339 >> 172.21.15.102:0/2861881339 conn(0x7f2ccc09e820 msgr2=0x7f2ccc09ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:13.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.130+0000 7f2cd279d640 1 -- 172.21.15.102:0/2861881339 shutdown_connections 2024-09-13T17:44:13.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.130+0000 7f2cd279d640 1 -- 172.21.15.102:0/2861881339 wait complete. 2024-09-13T17:44:13.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.131+0000 7f2cd279d640 1 Processor -- start 2024-09-13T17:44:13.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.131+0000 7f2cd279d640 1 -- start start 2024-09-13T17:44:13.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.131+0000 7f2cd279d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ccc09abb0 0x7f2ccc11aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.131+0000 7f2cd279d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ccc12c6a0 0x7f2ccc11af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.131+0000 7f2cd279d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ccc12cae0 0x7f2ccc13c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.131+0000 7f2cd279d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ccc09a600 con 0x7f2ccc12c6a0 2024-09-13T17:44:13.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.131+0000 7f2cd279d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ccc0981a0 con 0x7f2ccc09abb0 2024-09-13T17:44:13.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.131+0000 7f2cd279d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2ccc11c900 con 0x7f2ccc12cae0 2024-09-13T17:44:13.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.131+0000 7f2ccbfff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ccc09abb0 0x7f2ccc11aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.131+0000 7f2ccbfff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ccc09abb0 0x7f2ccc11aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35208/0 (socket says 172.21.15.102:35208) 2024-09-13T17:44:13.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.131+0000 7f2ccb7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ccc12c6a0 0x7f2ccc11af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.131+0000 7f2ccbfff640 1 -- 172.21.15.102:0/1943404586 learned_addr learned my addr 172.21.15.102:0/1943404586 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:13.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.131+0000 7f2cd0d3a640 1 --2- 172.21.15.102:0/1943404586 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ccc12cae0 0x7f2ccc13c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.132+0000 7f2ccbfff640 1 -- 172.21.15.102:0/1943404586 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ccc12cae0 msgr2=0x7f2ccc13c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:13.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.132+0000 7f2ccbfff640 1 --2- 172.21.15.102:0/1943404586 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ccc12cae0 0x7f2ccc13c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.132+0000 7f2ccbfff640 1 -- 172.21.15.102:0/1943404586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ccc12c6a0 msgr2=0x7f2ccc11af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:13.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.132+0000 7f2ccbfff640 1 --2- 172.21.15.102:0/1943404586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ccc12c6a0 0x7f2ccc11af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.132+0000 7f2ccbfff640 1 -- 172.21.15.102:0/1943404586 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cbc009390 con 0x7f2ccc09abb0 2024-09-13T17:44:13.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.132+0000 7f2ccb7fe640 1 --2- 172.21.15.102:0/1943404586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ccc12c6a0 0x7f2ccc11af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:13.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.132+0000 7f2ccbfff640 1 --2- 172.21.15.102:0/1943404586 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ccc09abb0 0x7f2ccc11aa00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00ec80 tx=0x7f2cbc0090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:13.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.132+0000 7f2cb3fff640 1 -- 172.21.15.102:0/1943404586 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc00eec0 con 0x7f2ccc09abb0 2024-09-13T17:44:13.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.132+0000 7f2cb3fff640 1 -- 172.21.15.102:0/1943404586 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cbc00cbc0 con 0x7f2ccc09abb0 2024-09-13T17:44:13.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.132+0000 7f2cd279d640 1 -- 172.21.15.102:0/1943404586 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cc4009a20 con 0x7f2ccc09abb0 2024-09-13T17:44:13.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.132+0000 7f2cb3fff640 1 -- 172.21.15.102:0/1943404586 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc013470 con 0x7f2ccc09abb0 2024-09-13T17:44:13.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.132+0000 7f2cd279d640 1 -- 172.21.15.102:0/1943404586 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ccc11b480 con 0x7f2ccc09abb0 2024-09-13T17:44:13.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.133+0000 7f2cb3fff640 1 -- 172.21.15.102:0/1943404586 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f2cbc0137e0 con 0x7f2ccc09abb0 2024-09-13T17:44:13.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.133+0000 7f2cb3fff640 1 -- 172.21.15.102:0/1943404586 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3660+0+0 (secure 0 0 0) 0x7f2cbc042800 con 0x7f2ccc09abb0 2024-09-13T17:44:13.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.135+0000 7f2cd4ca0740 1 Processor -- start 2024-09-13T17:44:13.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.135+0000 7f2cd4ca0740 1 -- start start 2024-09-13T17:44:13.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.135+0000 7f2cd4ca0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581bada1a90 0x5581bada1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.135+0000 7f2cd4ca0740 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5581bad87050 0x5581bada23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.135+0000 7f2cd4ca0740 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5581bad9c510 0x5581bad9c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.135+0000 7f2cd4ca0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581bacf1e70 con 0x5581bada1a90 2024-09-13T17:44:13.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.135+0000 7f2cd4ca0740 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x5581bacef600 con 0x5581bad87050 2024-09-13T17:44:13.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.135+0000 7f2cd4ca0740 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x5581baceb2e0 con 0x5581bad9c510 2024-09-13T17:44:13.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.135+0000 7f2ccbfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581bada1a90 0x5581bada1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.135+0000 7f2ccb7fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5581bad87050 0x5581bada23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.135+0000 7f2ccbfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581bada1a90 0x5581bada1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56948/0 (socket says 172.21.15.102:56948) 2024-09-13T17:44:13.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.135+0000 7f2ccbfff640 1 -- 172.21.15.102:0/2411249229 learned_addr learned my addr 172.21.15.102:0/2411249229 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:13.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.135+0000 7f2cd0d3a640 1 --2- 172.21.15.102:0/2411249229 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5581bad9c510 0x5581bad9c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.135+0000 7f2ccb7fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5581bad87050 0x5581bada23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35218/0 (socket says 172.21.15.102:35218) 2024-09-13T17:44:13.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.136+0000 7f2ccbfff640 1 -- 172.21.15.102:0/2411249229 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5581bad9c510 msgr2=0x5581bad9c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:13.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.136+0000 7f2ccbfff640 1 --2- 172.21.15.102:0/2411249229 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5581bad9c510 0x5581bad9c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.136+0000 7f2ccbfff640 1 -- 172.21.15.102:0/2411249229 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5581bad87050 msgr2=0x5581bada23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:13.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.136+0000 7f2ccbfff640 1 --2- 172.21.15.102:0/2411249229 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5581bad87050 0x5581bada23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.136+0000 7f2ccbfff640 1 -- 172.21.15.102:0/2411249229 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581baa139e0 con 0x5581bada1a90 2024-09-13T17:44:13.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.136+0000 7f2ccb7fe640 1 --2- 172.21.15.102:0/2411249229 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5581bad87050 0x5581bada23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:13.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.136+0000 7f2ccbfff640 1 --2- 172.21.15.102:0/2411249229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581bada1a90 0x5581bada1e60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc041690 tx=0x7f2cbc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90afb8b73e53ab80 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:13.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.136+0000 7f2cb17fa640 1 -- 172.21.15.102:0/2411249229 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc035070 con 0x5581bada1a90 2024-09-13T17:44:13.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.136+0000 7f2cb17fa640 1 -- 172.21.15.102:0/2411249229 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cbc01b3b0 con 0x5581bada1a90 2024-09-13T17:44:13.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.136+0000 7f2cb17fa640 1 -- 172.21.15.102:0/2411249229 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc01b930 con 0x5581bada1a90 2024-09-13T17:44:13.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.136+0000 7f2cd4ca0740 1 -- 172.21.15.102:0/2411249229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581bada1a90 msgr2=0x5581bada1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:13.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.136+0000 7f2cd4ca0740 1 --2- 172.21.15.102:0/2411249229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581bada1a90 0x5581bada1e60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc041690 tx=0x7f2cbc017000 comp rx=0 tx=0).stop 2024-09-13T17:44:13.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.137+0000 7f2cd4ca0740 1 -- 172.21.15.102:0/2411249229 shutdown_connections 2024-09-13T17:44:13.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.137+0000 7f2cd4ca0740 1 --2- 172.21.15.102:0/2411249229 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5581bad9c510 0x5581bad9c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.137+0000 7f2cd4ca0740 1 --2- 172.21.15.102:0/2411249229 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5581bad87050 0x5581bada23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.137+0000 7f2cd4ca0740 1 --2- 172.21.15.102:0/2411249229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581bada1a90 0x5581bada1e60 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.137+0000 7f2cd4ca0740 1 -- 172.21.15.102:0/2411249229 >> 172.21.15.102:0/2411249229 conn(0x5581bad87b30 msgr2=0x5581bad89300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:13.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.137+0000 7f2cd4ca0740 1 -- 172.21.15.102:0/2411249229 shutdown_connections 2024-09-13T17:44:13.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.137+0000 7f2cd4ca0740 1 -- 172.21.15.102:0/2411249229 wait complete. 2024-09-13T17:44:13.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.137+0000 7f2cd4ca0740 1 -- start start 2024-09-13T17:44:13.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.137+0000 7f2cd4ca0740 1 Processor -- start 2024-09-13T17:44:13.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.137+0000 7f2cd4ca0740 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5581bad87050 0x5581bada76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.137+0000 7f2cd4ca0740 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5581bad9c510 0x5581bada7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.137+0000 7f2cd4ca0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581bada1a90 0x5581bada8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.137+0000 7f2cd4ca0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581bac8d1b0 con 0x5581bada1a90 2024-09-13T17:44:13.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.137+0000 7f2cd4ca0740 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x5581bac8ce50 con 0x5581bad9c510 2024-09-13T17:44:13.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.137+0000 7f2cd4ca0740 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x5581baa35590 con 0x5581bad87050 2024-09-13T17:44:13.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.138+0000 7f2cd4ca0740 10 client.0 _ll_register_callbacks cb 0x5581bace4b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:44:13.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.138+0000 7f2ccb7fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5581bad9c510 0x5581bada7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.138+0000 7f2cd0d3a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581bada1a90 0x5581bada8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.138+0000 7f2ccbfff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5581bad87050 0x5581bada76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.138+0000 7f2ccb7fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5581bad9c510 0x5581bada7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35226/0 (socket says 172.21.15.102:35226) 2024-09-13T17:44:13.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.138+0000 7f2cd0d3a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581bada1a90 0x5581bada8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56954/0 (socket says 172.21.15.102:56954) 2024-09-13T17:44:13.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.138+0000 7f2ccb7fe640 1 -- 172.21.15.102:0/32249279 learned_addr learned my addr 172.21.15.102:0/32249279 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:13.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.138+0000 7f2ccb7fe640 1 -- 172.21.15.102:0/32249279 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5581bad87050 msgr2=0x5581bada76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:13.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.138+0000 7f2ccb7fe640 1 --2- 172.21.15.102:0/32249279 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5581bad87050 0x5581bada76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.138+0000 7f2ccb7fe640 1 -- 172.21.15.102:0/32249279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581bada1a90 msgr2=0x5581bada8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:13.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.138+0000 7f2ccb7fe640 1 --2- 172.21.15.102:0/32249279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581bada1a90 0x5581bada8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.138+0000 7f2ccb7fe640 1 -- 172.21.15.102:0/32249279 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ccc11b480 con 0x5581bad9c510 2024-09-13T17:44:13.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.138+0000 7f2ccbfff640 1 --2- 172.21.15.102:0/32249279 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5581bad87050 0x5581bada76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:13.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.138+0000 7f2cd0d3a640 1 --2- 172.21.15.102:0/32249279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581bada1a90 0x5581bada8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:13.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.138+0000 7f2ccb7fe640 1 --2- 172.21.15.102:0/32249279 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5581bad9c510 0x5581bada7be0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0014c40 tx=0x7f2cc0009040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d506b2a0017d4a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:13.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.138+0000 7f2caa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.102:3301/0 2024-09-13T17:44:13.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.138+0000 7f2caa7fc640 1 -- 172.21.15.102:0/32249279 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0014e80 con 0x5581bad9c510 2024-09-13T17:44:13.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.138+0000 7f2caa7fc640 1 -- 172.21.15.102:0/32249279 <== mon.1 v2:172.21.15.102:3301/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 3660+0+0 (secure 0 0 0) 0x7f2cc0011e30 con 0x5581bad9c510 2024-09-13T17:44:13.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.138+0000 7f2cd4ca0740 1 -- 172.21.15.102:0/32249279 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5581ba9f13c0 con 0x5581bad9c510 2024-09-13T17:44:13.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.138+0000 7f2caa7fc640 1 -- 172.21.15.102:0/32249279 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_get_version_reply(handle=1 version=11) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cc0017670 con 0x5581bad9c510 2024-09-13T17:44:13.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.138+0000 7f2cd4ca0740 10 client.5221 fetch_fsmap learned FSMap version 11 2024-09-13T17:44:13.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.138+0000 7f2cd4ca0740 1 -- 172.21.15.102:0/32249279 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,fsmap=11,monmap=2+}) v3 -- 0x5581bac9dab0 con 0x5581bad9c510 2024-09-13T17:44:13.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.138+0000 7f2caa7fc640 1 -- 172.21.15.102:0/32249279 <== mon.1 v2:172.21.15.102:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5581ba9f13c0 con 0x5581bad9c510 2024-09-13T17:44:13.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.138+0000 7f2caa7fc640 1 -- 172.21.15.102:0/32249279 <== mon.1 v2:172.21.15.102:3301/0 5 ==== fsmap(e 11) v1 ==== 1430+0+0 (secure 0 0 0) 0x7f2cc00203f0 con 0x5581bad9c510 2024-09-13T17:44:13.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.139+0000 7f2cd4ca0740 10 client.5221 fetch_fsmap finished waiting for FSMap version 11 2024-09-13T17:44:13.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.139+0000 7f2cd4ca0740 10 client.5221 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-13T17:44:13.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.139+0000 7f2cd4ca0740 20 client.5221 populate_metadata read hostname 'smithi102' 2024-09-13T17:44:13.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.139+0000 7f2cd4ca0740 4 client.5221 mds_command: new command op to 5078 tid=0[{"prefix": "get_command_descriptions"}] 2024-09-13T17:44:13.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.139+0000 7f2cd4ca0740 1 -- 172.21.15.102:0/32249279 --> v1:172.21.15.102:6817/2209249660 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5581baa026c0 con 0x5581badbe7a0 2024-09-13T17:44:13.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.139+0000 7f2caa7fc640 10 client.5221 ms_handle_connect on v1:172.21.15.102:6817/2209249660 2024-09-13T17:44:13.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.139+0000 7f2caa7fc640 1 -- 172.21.15.102:0/32249279 <== mds.0 v1:172.21.15.102:6817/2209249660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17536 (unknown 0 0 610023333) 0x5581baa026c0 con 0x5581badbe7a0 2024-09-13T17:44:13.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.139+0000 7f2caa7fc640 10 client.5221 handle_command_reply: tid=0 2024-09-13T17:44:13.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.139+0000 7f2cd4ca0740 1 client.5221 shutdown 2024-09-13T17:44:13.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.140+0000 7f2cd4ca0740 1 -- 172.21.15.102:0/32249279 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5581bad9c510 msgr2=0x5581bada7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:13.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.140+0000 7f2cd4ca0740 1 --2- 172.21.15.102:0/32249279 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5581bad9c510 0x5581bada7be0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0014c40 tx=0x7f2cc0009040 comp rx=0 tx=0).stop 2024-09-13T17:44:13.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.140+0000 7f2cd4ca0740 1 -- 172.21.15.102:0/32249279 shutdown_connections 2024-09-13T17:44:13.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.140+0000 7f2cd4ca0740 1 --2- 172.21.15.102:0/32249279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581bada1a90 0x5581bada8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.140+0000 7f2cd4ca0740 1 --2- 172.21.15.102:0/32249279 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5581bad9c510 0x5581bada7be0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.140+0000 7f2caa7fc640 0 client.5221 ms_handle_reset on v1:172.21.15.102:6817/2209249660 2024-09-13T17:44:13.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.140+0000 7f2cd4ca0740 1 --2- 172.21.15.102:0/32249279 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5581bad87050 0x5581bada76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.140+0000 7f2cd4ca0740 1 -- 172.21.15.102:0/32249279 >> 172.21.15.102:0/32249279 conn(0x5581bad87b30 msgr2=0x5581bada1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:13.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.140+0000 7f2cd4ca0740 1 -- 172.21.15.102:0/32249279 shutdown_connections 2024-09-13T17:44:13.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.140+0000 7f2cd4ca0740 1 -- 172.21.15.102:0/32249279 wait complete. 2024-09-13T17:44:13.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.140+0000 7f2cd4ca0740 20 client.5221 trim_cache size 0 max 16384 2024-09-13T17:44:13.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.154+0000 7f2cd4ca0740 1 Processor -- start 2024-09-13T17:44:13.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.154+0000 7f2cd4ca0740 1 -- start start 2024-09-13T17:44:13.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.154+0000 7f2cd4ca0740 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5581bad87050 0x5581badc1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.154+0000 7f2cd4ca0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581bad9c510 0x5581bad9f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.154+0000 7f2cd4ca0740 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5581bada1a90 0x5581bad9fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.154+0000 7f2cd4ca0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581bad81980 con 0x5581bad9c510 2024-09-13T17:44:13.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.154+0000 7f2cd4ca0740 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x5581bad87920 con 0x5581bada1a90 2024-09-13T17:44:13.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.154+0000 7f2cd4ca0740 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x5581bad89980 con 0x5581bad87050 2024-09-13T17:44:13.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.154+0000 7f2ccb7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581bad9c510 0x5581bad9f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.154+0000 7f2cd0d3a640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5581bada1a90 0x5581bad9fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.154+0000 7f2ccbfff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5581bad87050 0x5581badc1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.154+0000 7f2ccb7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581bad9c510 0x5581bad9f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56962/0 (socket says 172.21.15.102:56962) 2024-09-13T17:44:13.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.154+0000 7f2ccb7fe640 1 -- 172.21.15.102:0/2204356173 learned_addr learned my addr 172.21.15.102:0/2204356173 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:13.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.155+0000 7f2ccb7fe640 1 -- 172.21.15.102:0/2204356173 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5581bad87050 msgr2=0x5581badc1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:13.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.155+0000 7f2ccb7fe640 1 --2- 172.21.15.102:0/2204356173 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5581bad87050 0x5581badc1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.155+0000 7f2ccb7fe640 1 -- 172.21.15.102:0/2204356173 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5581bada1a90 msgr2=0x5581bad9fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:13.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.155+0000 7f2ccb7fe640 1 --2- 172.21.15.102:0/2204356173 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5581bada1a90 0x5581bad9fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.155+0000 7f2ccb7fe640 1 -- 172.21.15.102:0/2204356173 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581bac91bc0 con 0x5581bad9c510 2024-09-13T17:44:13.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.155+0000 7f2cd0d3a640 1 --2- 172.21.15.102:0/2204356173 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5581bada1a90 0x5581bad9fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:13.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.155+0000 7f2ccb7fe640 1 --2- 172.21.15.102:0/2204356173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581bad9c510 0x5581bad9f6b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0001150 tx=0x7f2cc001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be1aa3471543995c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:13.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.155+0000 7f2cb1ffb640 1 -- 172.21.15.102:0/2204356173 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0014e80 con 0x5581bad9c510 2024-09-13T17:44:13.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.155+0000 7f2cb1ffb640 1 -- 172.21.15.102:0/2204356173 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc00078b0 con 0x5581bad9c510 2024-09-13T17:44:13.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.155+0000 7f2cb1ffb640 1 -- 172.21.15.102:0/2204356173 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc00203f0 con 0x5581bad9c510 2024-09-13T17:44:13.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.155+0000 7f2cd4ca0740 1 -- 172.21.15.102:0/2204356173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581bad9c510 msgr2=0x5581bad9f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:13.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.155+0000 7f2cd4ca0740 1 --2- 172.21.15.102:0/2204356173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581bad9c510 0x5581bad9f6b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0001150 tx=0x7f2cc001d000 comp rx=0 tx=0).stop 2024-09-13T17:44:13.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.156+0000 7f2cd4ca0740 1 -- 172.21.15.102:0/2204356173 shutdown_connections 2024-09-13T17:44:13.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.156+0000 7f2cd4ca0740 1 --2- 172.21.15.102:0/2204356173 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5581bada1a90 0x5581bad9fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.156+0000 7f2cd4ca0740 1 --2- 172.21.15.102:0/2204356173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581bad9c510 0x5581bad9f6b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.156+0000 7f2cd4ca0740 1 --2- 172.21.15.102:0/2204356173 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5581bad87050 0x5581badc1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.156+0000 7f2cd4ca0740 1 -- 172.21.15.102:0/2204356173 >> 172.21.15.102:0/2204356173 conn(0x5581bad87b30 msgr2=0x5581bada34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:13.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.156+0000 7f2cd4ca0740 1 -- 172.21.15.102:0/2204356173 shutdown_connections 2024-09-13T17:44:13.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.156+0000 7f2cd4ca0740 1 -- 172.21.15.102:0/2204356173 wait complete. 2024-09-13T17:44:13.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.156+0000 7f2cd4ca0740 1 -- start start 2024-09-13T17:44:13.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.156+0000 7f2cd4ca0740 1 Processor -- start 2024-09-13T17:44:13.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.156+0000 7f2cd4ca0740 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5581bad87050 0x5581badef570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.156+0000 7f2cd4ca0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581bad9c510 0x5581badefab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.156+0000 7f2cd4ca0740 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5581bada1a90 0x5581bae0df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.156+0000 7f2cd4ca0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581badb0280 con 0x5581bad9c510 2024-09-13T17:44:13.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.156+0000 7f2cd4ca0740 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x5581badf09c0 con 0x5581bad87050 2024-09-13T17:44:13.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.156+0000 7f2cd4ca0740 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x5581badf1bd0 con 0x5581bada1a90 2024-09-13T17:44:13.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.157+0000 7f2cd4ca0740 10 client.0 _ll_register_callbacks cb 0x7f2cc0001d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:44:13.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.157+0000 7f2ccb7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581bad9c510 0x5581badefab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.157+0000 7f2ccbfff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5581bad87050 0x5581badef570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.157+0000 7f2cd0d3a640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5581bada1a90 0x5581bae0df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.157+0000 7f2ccb7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581bad9c510 0x5581badefab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56978/0 (socket says 172.21.15.102:56978) 2024-09-13T17:44:13.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.157+0000 7f2ccbfff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5581bad87050 0x5581badef570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35248/0 (socket says 172.21.15.102:35248) 2024-09-13T17:44:13.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.157+0000 7f2ccb7fe640 1 -- 172.21.15.102:0/3934836667 learned_addr learned my addr 172.21.15.102:0/3934836667 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:13.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.157+0000 7f2cd0d3a640 1 --2- 172.21.15.102:0/3934836667 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5581bada1a90 0x5581bae0df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:57758/0 (socket says 172.21.15.102:57758) 2024-09-13T17:44:13.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.157+0000 7f2ccb7fe640 1 -- 172.21.15.102:0/3934836667 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5581bada1a90 msgr2=0x5581bae0df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:13.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.157+0000 7f2ccb7fe640 1 --2- 172.21.15.102:0/3934836667 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5581bada1a90 0x5581bae0df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.157+0000 7f2ccb7fe640 1 -- 172.21.15.102:0/3934836667 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5581bad87050 msgr2=0x5581badef570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:13.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.157+0000 7f2ccb7fe640 1 --2- 172.21.15.102:0/3934836667 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5581bad87050 0x5581badef570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.157+0000 7f2ccb7fe640 1 -- 172.21.15.102:0/3934836667 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ccc098930 con 0x5581bad9c510 2024-09-13T17:44:13.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.157+0000 7f2cd0d3a640 1 --2- 172.21.15.102:0/3934836667 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5581bada1a90 0x5581bae0df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:13.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.157+0000 7f2ccbfff640 1 --2- 172.21.15.102:0/3934836667 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5581bad87050 0x5581badef570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:13.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.157+0000 7f2ccb7fe640 1 --2- 172.21.15.102:0/3934836667 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581bad9c510 0x5581badefab0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2cc00144e0 tx=0x7f2cc002b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3342dfe189e5d390 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:13.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.157+0000 7f2caa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-13T17:44:13.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.157+0000 7f2caa7fc640 1 -- 172.21.15.102:0/3934836667 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0014890 con 0x5581bad9c510 2024-09-13T17:44:13.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.157+0000 7f2caa7fc640 1 -- 172.21.15.102:0/3934836667 <== mon.0 v2:172.21.15.102:3300/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 3660+0+0 (secure 0 0 0) 0x7f2cc00203f0 con 0x5581bad9c510 2024-09-13T17:44:13.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.157+0000 7f2cd4ca0740 1 -- 172.21.15.102:0/3934836667 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2cbc01b3b0 con 0x5581bad9c510 2024-09-13T17:44:13.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.157+0000 7f2caa7fc640 1 -- 172.21.15.102:0/3934836667 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=11) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cc0017880 con 0x5581bad9c510 2024-09-13T17:44:13.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.158+0000 7f2cd4ca0740 10 client.5694 fetch_fsmap learned FSMap version 11 2024-09-13T17:44:13.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.158+0000 7f2cd4ca0740 1 -- 172.21.15.102:0/3934836667 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=11,monmap=2+}) v3 -- 0x5581bac9dab0 con 0x5581bad9c510 2024-09-13T17:44:13.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.158+0000 7f2caa7fc640 1 -- 172.21.15.102:0/3934836667 <== mon.0 v2:172.21.15.102:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cbc01b3b0 con 0x5581bad9c510 2024-09-13T17:44:13.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.158+0000 7f2caa7fc640 1 -- 172.21.15.102:0/3934836667 <== mon.0 v2:172.21.15.102:3300/0 5 ==== fsmap(e 11) v1 ==== 1430+0+0 (secure 0 0 0) 0x7f2cc003c440 con 0x5581bad9c510 2024-09-13T17:44:13.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.158+0000 7f2cd4ca0740 10 client.5694 fetch_fsmap finished waiting for FSMap version 11 2024-09-13T17:44:13.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.158+0000 7f2cd4ca0740 10 client.5694 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-13T17:44:13.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.158+0000 7f2cd4ca0740 20 client.5694 populate_metadata read hostname 'smithi102' 2024-09-13T17:44:13.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.158+0000 7f2cd4ca0740 4 client.5694 mds_command: new command op to 5078 tid=0[{"prefix": "status"}] 2024-09-13T17:44:13.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.158+0000 7f2cd4ca0740 1 -- 172.21.15.102:0/3934836667 --> v1:172.21.15.102:6817/2209249660 -- command(tid 0: {"prefix": "status"}) v1 -- 0x5581baa55d10 con 0x5581bae0e9b0 2024-09-13T17:44:13.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.158+0000 7f2caa7fc640 10 client.5694 ms_handle_connect on v1:172.21.15.102:6817/2209249660 2024-09-13T17:44:13.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.158+0000 7f2caa7fc640 1 -- 172.21.15.102:0/3934836667 <== mds.0 v1:172.21.15.102:6817/2209249660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (unknown 0 0 3593775812) 0x5581baa55d10 con 0x5581bae0e9b0 2024-09-13T17:44:13.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.158+0000 7f2caa7fc640 10 client.5694 handle_command_reply: tid=0 2024-09-13T17:44:13.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.158+0000 7f2cd4ca0740 1 client.5694 shutdown 2024-09-13T17:44:13.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.158+0000 7f2cd4ca0740 1 -- 172.21.15.102:0/3934836667 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581bad9c510 msgr2=0x5581badefab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:13.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.158+0000 7f2cd4ca0740 1 --2- 172.21.15.102:0/3934836667 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581bad9c510 0x5581badefab0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2cc00144e0 tx=0x7f2cc002b070 comp rx=0 tx=0).stop 2024-09-13T17:44:13.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.158+0000 7f2cd4ca0740 1 -- 172.21.15.102:0/3934836667 shutdown_connections 2024-09-13T17:44:13.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.158+0000 7f2cd4ca0740 1 --2- 172.21.15.102:0/3934836667 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x5581bada1a90 0x5581bae0df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.158+0000 7f2caa7fc640 0 client.5694 ms_handle_reset on v1:172.21.15.102:6817/2209249660 2024-09-13T17:44:13.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.158+0000 7f2cd4ca0740 1 --2- 172.21.15.102:0/3934836667 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581bad9c510 0x5581badefab0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.158+0000 7f2cd4ca0740 1 --2- 172.21.15.102:0/3934836667 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x5581bad87050 0x5581badef570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.159+0000 7f2cd4ca0740 1 -- 172.21.15.102:0/3934836667 >> 172.21.15.102:0/3934836667 conn(0x5581bad87b30 msgr2=0x5581bad84e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:13.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.159+0000 7f2cd4ca0740 1 -- 172.21.15.102:0/3934836667 shutdown_connections 2024-09-13T17:44:13.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.159+0000 7f2cd4ca0740 1 -- 172.21.15.102:0/3934836667 wait complete. 2024-09-13T17:44:13.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.159+0000 7f2cd4ca0740 20 client.5694 trim_cache size 0 max 16384 2024-09-13T17:44:13.171 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-13T17:44:13.171 INFO:teuthology.orchestra.run.smithi102.stdout: "cluster_fsid": "6507a4f0-6c1b-42c0-9c4d-5af24a57491d", 2024-09-13T17:44:13.171 INFO:teuthology.orchestra.run.smithi102.stdout: "whoami": 0, 2024-09-13T17:44:13.171 INFO:teuthology.orchestra.run.smithi102.stdout: "id": 5078, 2024-09-13T17:44:13.171 INFO:teuthology.orchestra.run.smithi102.stdout: "want_state": "up:active", 2024-09-13T17:44:13.172 INFO:teuthology.orchestra.run.smithi102.stdout: "state": "up:active", 2024-09-13T17:44:13.172 INFO:teuthology.orchestra.run.smithi102.stdout: "fs_name": "cephfs", 2024-09-13T17:44:13.172 INFO:teuthology.orchestra.run.smithi102.stdout: "rank_uptime": 4.086266588, 2024-09-13T17:44:13.172 INFO:teuthology.orchestra.run.smithi102.stdout: "mdsmap_epoch": 11, 2024-09-13T17:44:13.172 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_epoch": 32, 2024-09-13T17:44:13.172 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_epoch_barrier": 32, 2024-09-13T17:44:13.172 INFO:teuthology.orchestra.run.smithi102.stdout: "uptime": 11.979896624 2024-09-13T17:44:13.172 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-13T17:44:13.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.159+0000 7f2cd279d640 1 -- 172.21.15.102:0/1943404586 >> v1:172.21.15.102:6816/34937 conn(0x7f2cb8031160 legacy=0x7f2cb8033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:13.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.159+0000 7f2cd279d640 1 -- 172.21.15.102:0/1943404586 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ccc09abb0 msgr2=0x7f2ccc11aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:13.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.159+0000 7f2cd279d640 1 --2- 172.21.15.102:0/1943404586 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ccc09abb0 0x7f2ccc11aa00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00ec80 tx=0x7f2cbc0090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.159+0000 7f2cd279d640 1 -- 172.21.15.102:0/1943404586 shutdown_connections 2024-09-13T17:44:13.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.159+0000 7f2cd279d640 1 --2- 172.21.15.102:0/1943404586 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2ccc12cae0 0x7f2ccc13c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.159+0000 7f2cd279d640 1 --2- 172.21.15.102:0/1943404586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ccc12c6a0 0x7f2ccc11af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.159+0000 7f2cd279d640 1 --2- 172.21.15.102:0/1943404586 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2ccc09abb0 0x7f2ccc11aa00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.159+0000 7f2cd279d640 1 -- 172.21.15.102:0/1943404586 >> 172.21.15.102:0/1943404586 conn(0x7f2ccc09e820 msgr2=0x7f2ccc09c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:13.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.159+0000 7f2cd279d640 1 -- 172.21.15.102:0/1943404586 shutdown_connections 2024-09-13T17:44:13.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.159+0000 7f2cd279d640 1 -- 172.21.15.102:0/1943404586 wait complete. 2024-09-13T17:44:13.175 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T17:44:13.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.268+0000 7fa383e60640 1 Processor -- start 2024-09-13T17:44:13.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.268+0000 7fa383e60640 1 -- start start 2024-09-13T17:44:13.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.268+0000 7fa383e60640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa37c128330 0x7fa37c128700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.268+0000 7fa383e60640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa37c128c40 0x7fa37c12b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.268+0000 7fa383e60640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa37c12bd80 0x7fa37c12c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.268+0000 7fa383e60640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa37c12c760 con 0x7fa37c128c40 2024-09-13T17:44:13.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.268+0000 7fa383e60640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c097be0 con 0x7fa37c12bd80 2024-09-13T17:44:13.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.268+0000 7fa383e60640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c098e70 con 0x7fa37c128330 2024-09-13T17:44:13.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.269+0000 7fa381bfc640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa37c128330 0x7fa37c128700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.269+0000 7fa3823fd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa37c12bd80 0x7fa37c12c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.269+0000 7fa3813fb640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa37c128c40 0x7fa37c12b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.269+0000 7fa381bfc640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa37c128330 0x7fa37c128700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:57766/0 (socket says 172.21.15.102:57766) 2024-09-13T17:44:13.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.269+0000 7fa381bfc640 1 -- 172.21.15.102:0/2249939850 learned_addr learned my addr 172.21.15.102:0/2249939850 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:13.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.269+0000 7fa3823fd640 1 -- 172.21.15.102:0/2249939850 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa37c128330 msgr2=0x7fa37c128700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:13.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.269+0000 7fa3823fd640 1 --2- 172.21.15.102:0/2249939850 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa37c128330 0x7fa37c128700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.269+0000 7fa3823fd640 1 -- 172.21.15.102:0/2249939850 >> v2:172.21.15.102:3300/0 conn(0x7fa37c128c40 msgr2=0x7fa37c12b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:13.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.269+0000 7fa3823fd640 1 --2- 172.21.15.102:0/2249939850 >> v2:172.21.15.102:3300/0 conn(0x7fa37c128c40 0x7fa37c12b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.269+0000 7fa3823fd640 1 -- 172.21.15.102:0/2249939850 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa37c098310 con 0x7fa37c12bd80 2024-09-13T17:44:13.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.269+0000 7fa3813fb640 1 --2- 172.21.15.102:0/2249939850 >> v2:172.21.15.102:3300/0 conn(0x7fa37c128c40 0x7fa37c12b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:13.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.269+0000 7fa381bfc640 1 --2- 172.21.15.102:0/2249939850 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa37c128330 0x7fa37c128700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:13.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.270+0000 7fa3823fd640 1 --2- 172.21.15.102:0/2249939850 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa37c12bd80 0x7fa37c12c220 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa37400af70 tx=0x7fa37400a890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56aaf31a87d234a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:13.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.270+0000 7fa380bfa640 1 -- 172.21.15.102:0/2249939850 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37400a2e0 con 0x7fa37c12bd80 2024-09-13T17:44:13.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.270+0000 7fa380bfa640 1 -- 172.21.15.102:0/2249939850 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa374008450 con 0x7fa37c12bd80 2024-09-13T17:44:13.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.270+0000 7fa380bfa640 1 -- 172.21.15.102:0/2249939850 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa374008830 con 0x7fa37c12bd80 2024-09-13T17:44:13.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.270+0000 7fa383e60640 1 -- 172.21.15.102:0/2249939850 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa37c12bd80 msgr2=0x7fa37c12c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:13.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.270+0000 7fa383e60640 1 --2- 172.21.15.102:0/2249939850 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa37c12bd80 0x7fa37c12c220 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa37400af70 tx=0x7fa37400a890 comp rx=0 tx=0).stop 2024-09-13T17:44:13.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.270+0000 7fa383e60640 1 -- 172.21.15.102:0/2249939850 shutdown_connections 2024-09-13T17:44:13.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.270+0000 7fa383e60640 1 --2- 172.21.15.102:0/2249939850 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa37c12bd80 0x7fa37c12c220 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.270+0000 7fa383e60640 1 --2- 172.21.15.102:0/2249939850 >> v2:172.21.15.102:3300/0 conn(0x7fa37c128c40 0x7fa37c12b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.270+0000 7fa383e60640 1 --2- 172.21.15.102:0/2249939850 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa37c128330 0x7fa37c128700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.270+0000 7fa383e60640 1 -- 172.21.15.102:0/2249939850 >> 172.21.15.102:0/2249939850 conn(0x7fa37c09e820 msgr2=0x7fa37c09ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:13.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.270+0000 7fa383e60640 1 -- 172.21.15.102:0/2249939850 shutdown_connections 2024-09-13T17:44:13.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.270+0000 7fa383e60640 1 -- 172.21.15.102:0/2249939850 wait complete. 2024-09-13T17:44:13.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.271+0000 7fa383e60640 1 Processor -- start 2024-09-13T17:44:13.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.271+0000 7fa383e60640 1 -- start start 2024-09-13T17:44:13.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.272+0000 7fa383e60640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa37c128330 0x7fa37c119cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.272+0000 7fa383e60640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa37c128c40 0x7fa37c115e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.272+0000 7fa383e60640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa37c12bd80 0x7fa37c116370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.272+0000 7fa383e60640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c117e40 con 0x7fa37c12bd80 2024-09-13T17:44:13.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.272+0000 7fa383e60640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c09baf0 con 0x7fa37c128c40 2024-09-13T17:44:13.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.272+0000 7fa383e60640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c119230 con 0x7fa37c128330 2024-09-13T17:44:13.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.272+0000 7fa381bfc640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa37c128330 0x7fa37c119cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.272+0000 7fa381bfc640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa37c128330 0x7fa37c119cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:57774/0 (socket says 172.21.15.102:57774) 2024-09-13T17:44:13.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.272+0000 7fa381bfc640 1 -- 172.21.15.102:0/349000303 learned_addr learned my addr 172.21.15.102:0/349000303 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:13.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.272+0000 7fa3813fb640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa37c128c40 0x7fa37c115e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.272+0000 7fa3823fd640 1 --2- 172.21.15.102:0/349000303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa37c12bd80 0x7fa37c116370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.272+0000 7fa381bfc640 1 -- 172.21.15.102:0/349000303 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa37c128c40 msgr2=0x7fa37c115e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:13.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.272+0000 7fa381bfc640 1 --2- 172.21.15.102:0/349000303 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa37c128c40 0x7fa37c115e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.272+0000 7fa381bfc640 1 -- 172.21.15.102:0/349000303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa37c12bd80 msgr2=0x7fa37c116370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:13.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.272+0000 7fa381bfc640 1 --2- 172.21.15.102:0/349000303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa37c12bd80 0x7fa37c116370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.272+0000 7fa381bfc640 1 -- 172.21.15.102:0/349000303 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa378009450 con 0x7fa37c128330 2024-09-13T17:44:13.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.272+0000 7fa381bfc640 1 --2- 172.21.15.102:0/349000303 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa37c128330 0x7fa37c119cd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa37800ec80 tx=0x7fa3780091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:13.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.272+0000 7fa371ffb640 1 -- 172.21.15.102:0/349000303 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa378009730 con 0x7fa37c128330 2024-09-13T17:44:13.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.272+0000 7fa371ffb640 1 -- 172.21.15.102:0/349000303 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa37800eec0 con 0x7fa37c128330 2024-09-13T17:44:13.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.272+0000 7fa371ffb640 1 -- 172.21.15.102:0/349000303 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37800cdf0 con 0x7fa37c128330 2024-09-13T17:44:13.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.273+0000 7fa383e60640 1 -- 172.21.15.102:0/349000303 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa36c009390 con 0x7fa37c128330 2024-09-13T17:44:13.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.273+0000 7fa383e60640 1 -- 172.21.15.102:0/349000303 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa37400ab30 con 0x7fa37c128330 2024-09-13T17:44:13.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.274+0000 7fa371ffb640 1 -- 172.21.15.102:0/349000303 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fa378013680 con 0x7fa37c128330 2024-09-13T17:44:13.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.274+0000 7fa371ffb640 1 -- 172.21.15.102:0/349000303 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3660+0+0 (secure 0 0 0) 0x7fa378042410 con 0x7fa37c128330 2024-09-13T17:44:13.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.274+0000 7fa383e60640 1 -- 172.21.15.102:0/349000303 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa37c064000 con 0x7fa37c128330 2024-09-13T17:44:13.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.278+0000 7fa371ffb640 1 -- 172.21.15.102:0/349000303 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fa378021080 con 0x7fa37c128330 2024-09-13T17:44:13.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.465+0000 7fa383e60640 1 -- 172.21.15.102:0/349000303 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa37c1178c0 con 0x7fa37c128330 2024-09-13T17:44:13.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.465+0000 7fa371ffb640 1 -- 172.21.15.102:0/349000303 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v11) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa3780236b0 con 0x7fa37c128330 2024-09-13T17:44:13.467 INFO:teuthology.orchestra.run.smithi102.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T17:44:13.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.467+0000 7fa383e60640 1 -- 172.21.15.102:0/349000303 >> v1:172.21.15.102:6816/34937 conn(0x7fa360031160 legacy=0x7fa360033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:13.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.467+0000 7fa383e60640 1 -- 172.21.15.102:0/349000303 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa37c128330 msgr2=0x7fa37c119cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:13.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.467+0000 7fa383e60640 1 --2- 172.21.15.102:0/349000303 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa37c128330 0x7fa37c119cd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa37800ec80 tx=0x7fa3780091d0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.467+0000 7fa383e60640 1 -- 172.21.15.102:0/349000303 shutdown_connections 2024-09-13T17:44:13.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.467+0000 7fa383e60640 1 --2- 172.21.15.102:0/349000303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa37c12bd80 0x7fa37c116370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.467+0000 7fa383e60640 1 --2- 172.21.15.102:0/349000303 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa37c128c40 0x7fa37c115e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.467+0000 7fa383e60640 1 --2- 172.21.15.102:0/349000303 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa37c128330 0x7fa37c119cd0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.467+0000 7fa383e60640 1 -- 172.21.15.102:0/349000303 >> 172.21.15.102:0/349000303 conn(0x7fa37c09e820 msgr2=0x7fa37c09d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:13.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.467+0000 7fa383e60640 1 -- 172.21.15.102:0/349000303 shutdown_connections 2024-09-13T17:44:13.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:13.467+0000 7fa383e60640 1 -- 172.21.15.102:0/349000303 wait complete. 2024-09-13T17:44:13.480 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T17:44:13.480 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T17:44:13.480 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi171.front.sepia.ceph.com 2024-09-13T17:44:13.480 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-13T17:44:13.480 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T17:44:13.480 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T17:44:13.480 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T17:44:13.480 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T17:44:13.481 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T17:44:13.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:13.481 DEBUG:teuthology.orchestra.run.smithi171:> ip addr 2024-09-13T17:44:13.501 INFO:teuthology.orchestra.run.smithi171.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T17:44:13.501 INFO:teuthology.orchestra.run.smithi171.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T17:44:13.502 INFO:teuthology.orchestra.run.smithi171.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T17:44:13.502 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:13.502 INFO:teuthology.orchestra.run.smithi171.stdout: inet6 ::1/128 scope host 2024-09-13T17:44:13.502 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:13.502 INFO:teuthology.orchestra.run.smithi171.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T17:44:13.502 INFO:teuthology.orchestra.run.smithi171.stdout: link/ether 0c:c4:7a:d9:39:a2 brd ff:ff:ff:ff:ff:ff 2024-09-13T17:44:13.502 INFO:teuthology.orchestra.run.smithi171.stdout: altname enp7s0f0 2024-09-13T17:44:13.503 INFO:teuthology.orchestra.run.smithi171.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T17:44:13.503 INFO:teuthology.orchestra.run.smithi171.stdout: link/ether 0c:c4:7a:d9:39:a3 brd ff:ff:ff:ff:ff:ff 2024-09-13T17:44:13.503 INFO:teuthology.orchestra.run.smithi171.stdout: altname enp7s0f1 2024-09-13T17:44:13.503 INFO:teuthology.orchestra.run.smithi171.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T17:44:13.503 INFO:teuthology.orchestra.run.smithi171.stdout: link/ether 0c:c4:7a:88:6f:6c brd ff:ff:ff:ff:ff:ff 2024-09-13T17:44:13.503 INFO:teuthology.orchestra.run.smithi171.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T17:44:13.503 INFO:teuthology.orchestra.run.smithi171.stdout: link/ether 0c:c4:7a:88:6f:6d brd ff:ff:ff:ff:ff:ff 2024-09-13T17:44:13.504 INFO:teuthology.orchestra.run.smithi171.stdout: inet 172.21.15.171/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-13T17:44:13.504 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft 42664sec preferred_lft 42664sec 2024-09-13T17:44:13.504 INFO:teuthology.orchestra.run.smithi171.stdout: inet6 fe80::ec4:7aff:fe88:6f6d/64 scope link noprefixroute 2024-09-13T17:44:13.504 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:13.504 INFO:teuthology.orchestra.run.smithi171.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T17:44:13.504 INFO:teuthology.orchestra.run.smithi171.stdout: link/ether 5e:4f:35:a0:e0:0f brd ff:ff:ff:ff:ff:ff 2024-09-13T17:44:13.504 INFO:teuthology.orchestra.run.smithi171.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-13T17:44:13.505 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:13.505 INFO:teuthology.orchestra.run.smithi171.stdout: inet6 fe80::84b3:ccff:fec6:5b6a/64 scope link 2024-09-13T17:44:13.505 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:13.505 INFO:teuthology.orchestra.run.smithi171.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T17:44:13.505 INFO:teuthology.orchestra.run.smithi171.stdout: link/ether 5e:4f:35:a0:e0:0f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T17:44:13.505 INFO:teuthology.orchestra.run.smithi171.stdout: inet6 fe80::5c4f:35ff:fea0:e00f/64 scope link 2024-09-13T17:44:13.505 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:13.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:13.506 DEBUG:teuthology.orchestra.run.smithi171:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T17:44:13.569 INFO:teuthology.orchestra.run.smithi171.stdout:1 2024-09-13T17:44:13.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:13.571 DEBUG:teuthology.orchestra.run.smithi171:> route 2024-09-13T17:44:13.630 INFO:teuthology.orchestra.run.smithi171.stdout:Kernel IP routing table 2024-09-13T17:44:13.630 INFO:teuthology.orchestra.run.smithi171.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T17:44:13.630 INFO:teuthology.orchestra.run.smithi171.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-13T17:44:13.630 INFO:teuthology.orchestra.run.smithi171.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-13T17:44:13.630 INFO:teuthology.orchestra.run.smithi171.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T17:44:13.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:13.631 DEBUG:teuthology.orchestra.run.smithi171:> (cd / && exec bash -c ' 2024-09-13T17:44:13.631 DEBUG:teuthology.orchestra.run.smithi171:> set -e 2024-09-13T17:44:13.631 DEBUG:teuthology.orchestra.run.smithi171:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-13T17:44:13.631 DEBUG:teuthology.orchestra.run.smithi171:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-13T17:44:13.631 DEBUG:teuthology.orchestra.run.smithi171:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-13T17:44:13.631 DEBUG:teuthology.orchestra.run.smithi171:> ') 2024-09-13T17:44:13.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:13.722 DEBUG:teuthology.orchestra.run.smithi171:> ip netns list 2024-09-13T17:44:13.738 INFO:teuthology.orchestra.run.smithi171.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T17:44:13.738 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T17:44:13.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:13.739 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip link set brx.0 up 2024-09-13T17:44:13.805 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-13T17:44:13.805 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:44:13.805 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-13T17:44:13.860 INFO:teuthology.orchestra.run.smithi171.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-13T17:44:13.861 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:44:13.861 DEBUG:teuthology.orchestra.run.smithi171:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-13T17:44:13.915 DEBUG:teuthology.orchestra.run.smithi171:> sudo modprobe fuse 2024-09-13T17:44:13.981 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:44:13.981 DEBUG:teuthology.orchestra.run.smithi171:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:44:14.045 INFO:teuthology.orchestra.run.smithi171.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:44:14.046 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:44:14.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:14.047 DEBUG:teuthology.orchestra.run.smithi171:> ls /sys/fs/fuse/connections 2024-09-13T17:44:14.104 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-13T17:44:14.104 DEBUG:teuthology.orchestra.run.smithi171:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-13T17:44:14.147 DEBUG:teuthology.orchestra.run.smithi171:> sudo modprobe fuse 2024-09-13T17:44:14.171 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:44:14.171 DEBUG:teuthology.orchestra.run.smithi171:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:44:14.230 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi171.stderr:2024-09-13T17:44:14.229+0000 7ff9f1d20480 -1 init, newargv = 0x55d8df5404a0 newargc=15 2024-09-13T17:44:14.230 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi171.stderr:ceph-fuse[31022]: starting ceph client 2024-09-13T17:44:14.235 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi171.stderr:ceph-fuse[31022]: starting fuse 2024-09-13T17:44:14.236 INFO:teuthology.orchestra.run.smithi171.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:44:14.238 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:44:14.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:14.239 DEBUG:teuthology.orchestra.run.smithi171:> ls /sys/fs/fuse/connections 2024-09-13T17:44:14.294 INFO:teuthology.orchestra.run.smithi171.stdout:43 2024-09-13T17:44:14.294 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-09-13T17:44:14.294 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:44:14.294 DEBUG:teuthology.orchestra.run.smithi171:> import glob 2024-09-13T17:44:14.294 DEBUG:teuthology.orchestra.run.smithi171:> import re 2024-09-13T17:44:14.294 DEBUG:teuthology.orchestra.run.smithi171:> import os 2024-09-13T17:44:14.294 DEBUG:teuthology.orchestra.run.smithi171:> import subprocess 2024-09-13T17:44:14.295 DEBUG:teuthology.orchestra.run.smithi171:> 2024-09-13T17:44:14.295 DEBUG:teuthology.orchestra.run.smithi171:> def _find_admin_socket(client_name): 2024-09-13T17:44:14.295 DEBUG:teuthology.orchestra.run.smithi171:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-13T17:44:14.295 DEBUG:teuthology.orchestra.run.smithi171:> files = glob.glob(asok_path) 2024-09-13T17:44:14.295 DEBUG:teuthology.orchestra.run.smithi171:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-13T17:44:14.295 DEBUG:teuthology.orchestra.run.smithi171:> 2024-09-13T17:44:14.295 DEBUG:teuthology.orchestra.run.smithi171:> # Given a non-glob path, it better be there 2024-09-13T17:44:14.295 DEBUG:teuthology.orchestra.run.smithi171:> if "*" not in asok_path: 2024-09-13T17:44:14.295 DEBUG:teuthology.orchestra.run.smithi171:> assert(len(files) == 1) 2024-09-13T17:44:14.295 DEBUG:teuthology.orchestra.run.smithi171:> return files[0] 2024-09-13T17:44:14.295 DEBUG:teuthology.orchestra.run.smithi171:> 2024-09-13T17:44:14.296 DEBUG:teuthology.orchestra.run.smithi171:> for f in files: 2024-09-13T17:44:14.296 DEBUG:teuthology.orchestra.run.smithi171:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:44:14.296 DEBUG:teuthology.orchestra.run.smithi171:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:44:14.296 DEBUG:teuthology.orchestra.run.smithi171:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:44:14.296 DEBUG:teuthology.orchestra.run.smithi171:> contents = proc_f.read() 2024-09-13T17:44:14.296 DEBUG:teuthology.orchestra.run.smithi171:> if mountpoint in contents: 2024-09-13T17:44:14.296 DEBUG:teuthology.orchestra.run.smithi171:> return f 2024-09-13T17:44:14.296 DEBUG:teuthology.orchestra.run.smithi171:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:44:14.296 DEBUG:teuthology.orchestra.run.smithi171:> 2024-09-13T17:44:14.296 DEBUG:teuthology.orchestra.run.smithi171:> print(_find_admin_socket("client.0")) 2024-09-13T17:44:14.296 DEBUG:teuthology.orchestra.run.smithi171:> ' 2024-09-13T17:44:14.409 INFO:teuthology.orchestra.run.smithi171.stdout:/var/run/ceph/ceph-client.0.31022.asok 2024-09-13T17:44:14.595 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.31022.asok 2024-09-13T17:44:14.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:14.595 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.31022.asok status 2024-09-13T17:44:14.729 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-13T17:44:14.730 INFO:teuthology.orchestra.run.smithi171.stdout: "metadata": { 2024-09-13T17:44:14.730 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_sha1": "6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3", 2024-09-13T17:44:14.730 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T17:44:14.730 INFO:teuthology.orchestra.run.smithi171.stdout: "entity_id": "0", 2024-09-13T17:44:14.730 INFO:teuthology.orchestra.run.smithi171.stdout: "hostname": "smithi171", 2024-09-13T17:44:14.730 INFO:teuthology.orchestra.run.smithi171.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-13T17:44:14.730 INFO:teuthology.orchestra.run.smithi171.stdout: "pid": "31022", 2024-09-13T17:44:14.731 INFO:teuthology.orchestra.run.smithi171.stdout: "root": "/" 2024-09-13T17:44:14.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-13T17:44:14.731 INFO:teuthology.orchestra.run.smithi171.stdout: "dentry_count": 0, 2024-09-13T17:44:14.731 INFO:teuthology.orchestra.run.smithi171.stdout: "dentry_pinned_count": 0, 2024-09-13T17:44:14.731 INFO:teuthology.orchestra.run.smithi171.stdout: "id": 5239, 2024-09-13T17:44:14.731 INFO:teuthology.orchestra.run.smithi171.stdout: "inst": { 2024-09-13T17:44:14.731 INFO:teuthology.orchestra.run.smithi171.stdout: "name": { 2024-09-13T17:44:14.731 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "client", 2024-09-13T17:44:14.731 INFO:teuthology.orchestra.run.smithi171.stdout: "num": 5239 2024-09-13T17:44:14.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-13T17:44:14.731 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": { 2024-09-13T17:44:14.731 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-13T17:44:14.732 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:0", 2024-09-13T17:44:14.732 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2238967350 2024-09-13T17:44:14.732 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-13T17:44:14.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-13T17:44:14.732 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": { 2024-09-13T17:44:14.732 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-13T17:44:14.732 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "172.21.15.171:0", 2024-09-13T17:44:14.732 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 2238967350 2024-09-13T17:44:14.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-13T17:44:14.732 INFO:teuthology.orchestra.run.smithi171.stdout: "inst_str": "client.5239 172.21.15.171:0/2238967350", 2024-09-13T17:44:14.732 INFO:teuthology.orchestra.run.smithi171.stdout: "addr_str": "172.21.15.171:0/2238967350", 2024-09-13T17:44:14.732 INFO:teuthology.orchestra.run.smithi171.stdout: "inode_count": 1, 2024-09-13T17:44:14.733 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_epoch": 11, 2024-09-13T17:44:14.733 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_epoch": 32, 2024-09-13T17:44:14.733 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_epoch_barrier": 0, 2024-09-13T17:44:14.733 INFO:teuthology.orchestra.run.smithi171.stdout: "blocklisted": false, 2024-09-13T17:44:14.733 INFO:teuthology.orchestra.run.smithi171.stdout: "fs_name": "cephfs" 2024-09-13T17:44:14.733 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-13T17:44:14.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:14.738 DEBUG:teuthology.orchestra.run.smithi171:> stat --file-system '--printf=%T 2024-09-13T17:44:14.738 DEBUG:teuthology.orchestra.run.smithi171:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-13T17:44:14.754 INFO:teuthology.orchestra.run.smithi171.stdout:fuseblk 2024-09-13T17:44:14.755 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-13T17:44:14.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:14.755 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-13T17:44:14.820 INFO:tasks.cephfs.test_newops:Test for new getvxattr op... 2024-09-13T17:44:14.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:14.820 DEBUG:teuthology.orchestra.run.smithi171:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir newop_getvxattr_dir) 2024-09-13T17:44:14.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:14.883 DEBUG:teuthology.orchestra.run.smithi171:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.pin.random ./newop_getvxattr_dir) 2024-09-13T17:44:14.940 INFO:teuthology.orchestra.run.smithi171.stdout:0 2024-09-13T17:44:14.940 INFO:tasks.cephfs.test_newops:Test for new getvxattr op succeeds 2024-09-13T17:44:14.940 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T17:44:14.940 DEBUG:teuthology.orchestra.run.smithi102:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T17:44:14.972 DEBUG:teuthology.orchestra.run.smithi171:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T17:44:15.022 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T17:44:15.023 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T17:44:15.023 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:44:15.079 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:44:15.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.180+0000 7f194459d640 1 Processor -- start 2024-09-13T17:44:15.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.180+0000 7f194459d640 1 -- start start 2024-09-13T17:44:15.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.181+0000 7f194459d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f193c067a40 0x7f193c12f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:15.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.181+0000 7f194459d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f193c12fab0 0x7f193c12fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:15.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.181+0000 7f194459d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f193c09e7e0 0x7f193c09ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:15.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.181+0000 7f194459d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f193c067e10 con 0x7f193c067a40 2024-09-13T17:44:15.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.181+0000 7f194459d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f193c0690b0 con 0x7f193c09e7e0 2024-09-13T17:44:15.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.181+0000 7f194459d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f193c12f340 con 0x7f193c12fab0 2024-09-13T17:44:15.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.181+0000 7f1942b3a640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f193c09e7e0 0x7f193c09ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:15.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.181+0000 7f1941b38640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f193c12fab0 0x7f193c12fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:15.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.181+0000 7f1942b3a640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f193c09e7e0 0x7f193c09ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35272/0 (socket says 172.21.15.102:35272) 2024-09-13T17:44:15.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.181+0000 7f1942b3a640 1 -- 172.21.15.102:0/1715210188 learned_addr learned my addr 172.21.15.102:0/1715210188 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:15.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.181+0000 7f1942339640 1 --2- 172.21.15.102:0/1715210188 >> v2:172.21.15.102:3300/0 conn(0x7f193c067a40 0x7f193c12f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:15.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.181+0000 7f1942b3a640 1 -- 172.21.15.102:0/1715210188 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f193c12fab0 msgr2=0x7f193c12fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:15.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.181+0000 7f1942b3a640 1 --2- 172.21.15.102:0/1715210188 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f193c12fab0 0x7f193c12fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:15.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.181+0000 7f1942b3a640 1 -- 172.21.15.102:0/1715210188 >> v2:172.21.15.102:3300/0 conn(0x7f193c067a40 msgr2=0x7f193c12f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:15.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.181+0000 7f1942b3a640 1 --2- 172.21.15.102:0/1715210188 >> v2:172.21.15.102:3300/0 conn(0x7f193c067a40 0x7f193c12f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:15.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.181+0000 7f1942b3a640 1 -- 172.21.15.102:0/1715210188 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f193c068240 con 0x7f193c09e7e0 2024-09-13T17:44:15.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.181+0000 7f1941b38640 1 --2- 172.21.15.102:0/1715210188 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f193c12fab0 0x7f193c12fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:15.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.181+0000 7f1942339640 1 --2- 172.21.15.102:0/1715210188 >> v2:172.21.15.102:3300/0 conn(0x7f193c067a40 0x7f193c12f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:15.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.182+0000 7f1942b3a640 1 --2- 172.21.15.102:0/1715210188 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f193c09e7e0 0x7f193c09ec80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f192c009940 tx=0x7f192c0091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9837077128b9a2fd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:15.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.182+0000 7f1941337640 1 -- 172.21.15.102:0/1715210188 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c02ac70 con 0x7f193c09e7e0 2024-09-13T17:44:15.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.182+0000 7f1941337640 1 -- 172.21.15.102:0/1715210188 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192c02add0 con 0x7f193c09e7e0 2024-09-13T17:44:15.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.182+0000 7f1941337640 1 -- 172.21.15.102:0/1715210188 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c032670 con 0x7f193c09e7e0 2024-09-13T17:44:15.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.183+0000 7f194459d640 1 -- 172.21.15.102:0/1715210188 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f193c09e7e0 msgr2=0x7f193c09ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:15.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.183+0000 7f194459d640 1 --2- 172.21.15.102:0/1715210188 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f193c09e7e0 0x7f193c09ec80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f192c009940 tx=0x7f192c0091d0 comp rx=0 tx=0).stop 2024-09-13T17:44:15.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.183+0000 7f194459d640 1 -- 172.21.15.102:0/1715210188 shutdown_connections 2024-09-13T17:44:15.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.183+0000 7f194459d640 1 --2- 172.21.15.102:0/1715210188 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f193c09e7e0 0x7f193c09ec80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:15.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.183+0000 7f194459d640 1 --2- 172.21.15.102:0/1715210188 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f193c12fab0 0x7f193c12fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:15.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.183+0000 7f194459d640 1 --2- 172.21.15.102:0/1715210188 >> v2:172.21.15.102:3300/0 conn(0x7f193c067a40 0x7f193c12f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:15.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.183+0000 7f194459d640 1 -- 172.21.15.102:0/1715210188 >> 172.21.15.102:0/1715210188 conn(0x7f193c0a13a0 msgr2=0x7f193c067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:15.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.183+0000 7f194459d640 1 -- 172.21.15.102:0/1715210188 shutdown_connections 2024-09-13T17:44:15.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.183+0000 7f194459d640 1 -- 172.21.15.102:0/1715210188 wait complete. 2024-09-13T17:44:15.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.184+0000 7f194459d640 1 Processor -- start 2024-09-13T17:44:15.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.184+0000 7f194459d640 1 -- start start 2024-09-13T17:44:15.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.184+0000 7f194459d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f193c067a40 0x7f193c13e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:15.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.184+0000 7f194459d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f193c09e7e0 0x7f193c13eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:15.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.184+0000 7f194459d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f193c12fab0 0x7f193c13f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:15.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.184+0000 7f194459d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193c11d820 con 0x7f193c12fab0 2024-09-13T17:44:15.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.184+0000 7f194459d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f193c09db50 con 0x7f193c067a40 2024-09-13T17:44:15.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.184+0000 7f194459d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f193c11da50 con 0x7f193c09e7e0 2024-09-13T17:44:15.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.184+0000 7f1942339640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f193c067a40 0x7f193c13e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:15.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.184+0000 7f1942b3a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f193c12fab0 0x7f193c13f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:15.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.184+0000 7f1942339640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f193c067a40 0x7f193c13e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35278/0 (socket says 172.21.15.102:35278) 2024-09-13T17:44:15.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.184+0000 7f1942339640 1 -- 172.21.15.102:0/911145041 learned_addr learned my addr 172.21.15.102:0/911145041 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:15.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.184+0000 7f1941b38640 1 --2- 172.21.15.102:0/911145041 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f193c09e7e0 0x7f193c13eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:15.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.185+0000 7f1942b3a640 1 -- 172.21.15.102:0/911145041 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f193c09e7e0 msgr2=0x7f193c13eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:15.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.185+0000 7f1942b3a640 1 --2- 172.21.15.102:0/911145041 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f193c09e7e0 0x7f193c13eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:15.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.185+0000 7f1942b3a640 1 -- 172.21.15.102:0/911145041 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f193c067a40 msgr2=0x7f193c13e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:15.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.185+0000 7f1942b3a640 1 --2- 172.21.15.102:0/911145041 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f193c067a40 0x7f193c13e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:15.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.185+0000 7f1942b3a640 1 -- 172.21.15.102:0/911145041 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1934009390 con 0x7f193c12fab0 2024-09-13T17:44:15.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.185+0000 7f1942b3a640 1 --2- 172.21.15.102:0/911145041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f193c12fab0 0x7f193c13f2e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f193c06efc0 tx=0x7f192c032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:15.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.185+0000 7f193a7fc640 1 -- 172.21.15.102:0/911145041 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c02ac70 con 0x7f193c12fab0 2024-09-13T17:44:15.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.185+0000 7f193a7fc640 1 -- 172.21.15.102:0/911145041 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192c032e40 con 0x7f193c12fab0 2024-09-13T17:44:15.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.185+0000 7f194459d640 1 -- 172.21.15.102:0/911145041 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1930009a20 con 0x7f193c12fab0 2024-09-13T17:44:15.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.185+0000 7f193a7fc640 1 -- 172.21.15.102:0/911145041 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c031cd0 con 0x7f193c12fab0 2024-09-13T17:44:15.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.185+0000 7f194459d640 1 -- 172.21.15.102:0/911145041 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f192c009450 con 0x7f193c12fab0 2024-09-13T17:44:15.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.186+0000 7f193a7fc640 1 -- 172.21.15.102:0/911145041 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f192c039040 con 0x7f193c12fab0 2024-09-13T17:44:15.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.186+0000 7f193a7fc640 1 -- 172.21.15.102:0/911145041 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3660+0+0 (secure 0 0 0) 0x7f192c0619f0 con 0x7f193c12fab0 2024-09-13T17:44:15.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.186+0000 7f194459d640 1 -- 172.21.15.102:0/911145041 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f193c064000 con 0x7f193c12fab0 2024-09-13T17:44:15.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.191+0000 7f193a7fc640 1 -- 172.21.15.102:0/911145041 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f192c037080 con 0x7f193c12fab0 2024-09-13T17:44:15.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.343+0000 7f194459d640 1 -- 172.21.15.102:0/911145041 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f193c05e0d0 con 0x7f193c12fab0 2024-09-13T17:44:15.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.343+0000 7f193a7fc640 1 -- 172.21.15.102:0/911145041 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+11330 (secure 0 0 0) 0x7f192c03bc70 con 0x7f193c12fab0 2024-09-13T17:44:15.345 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:44:15.345 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":32,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:44:10.068600+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:44:06.718328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-13T17:44:07.355624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:44:15.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.345+0000 7f194459d640 1 -- 172.21.15.102:0/911145041 >> v1:172.21.15.102:6816/34937 conn(0x7f1928031160 legacy=0x7f1928033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:15.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.345+0000 7f194459d640 1 -- 172.21.15.102:0/911145041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f193c12fab0 msgr2=0x7f193c13f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:15.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.345+0000 7f194459d640 1 --2- 172.21.15.102:0/911145041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f193c12fab0 0x7f193c13f2e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f193c06efc0 tx=0x7f192c032000 comp rx=0 tx=0).stop 2024-09-13T17:44:15.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.345+0000 7f194459d640 1 -- 172.21.15.102:0/911145041 shutdown_connections 2024-09-13T17:44:15.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.345+0000 7f194459d640 1 --2- 172.21.15.102:0/911145041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f193c12fab0 0x7f193c13f2e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:15.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.345+0000 7f194459d640 1 --2- 172.21.15.102:0/911145041 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f193c09e7e0 0x7f193c13eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:15.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.345+0000 7f194459d640 1 --2- 172.21.15.102:0/911145041 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f193c067a40 0x7f193c13e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:15.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.345+0000 7f194459d640 1 -- 172.21.15.102:0/911145041 >> 172.21.15.102:0/911145041 conn(0x7f193c0a13a0 msgr2=0x7f193c130480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:15.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.345+0000 7f194459d640 1 -- 172.21.15.102:0/911145041 shutdown_connections 2024-09-13T17:44:15.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.345+0000 7f194459d640 1 -- 172.21.15.102:0/911145041 wait complete. 2024-09-13T17:44:15.358 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:54.442612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T17:41:59.076782+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 16, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:44:06.718328+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:44:07.355624+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:44:15.358 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:44:15.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.453+0000 7ff08fb5d640 1 Processor -- start 2024-09-13T17:44:15.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.453+0000 7ff08fb5d640 1 -- start start 2024-09-13T17:44:15.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.453+0000 7ff08fb5d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff088096a70 0x7ff0880987c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:15.456 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.453+0000 7ff08fb5d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff088099110 0x7ff08812c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:15.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.453+0000 7ff08fb5d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0880999e0 con 0x7ff088098d00 2024-09-13T17:44:15.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.453+0000 7ff08fb5d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff088069c30 con 0x7ff088099110 2024-09-13T17:44:15.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.453+0000 7ff08fb5d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff08806ed00 con 0x7ff088096a70 2024-09-13T17:44:15.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.454+0000 7ff08e0fa640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff088099110 0x7ff08812c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:15.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.454+0000 7ff08d8f9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff088096a70 0x7ff0880987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:15.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.454+0000 7ff08e0fa640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7ff088099110 0x7ff08812c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57030/0 (socket says 172.21.15.102:57030) 2024-09-13T17:44:15.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.454+0000 7ff08d0f8640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7ff088098d00 0x7ff08812c2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57208/0 (socket says 172.21.15.102:57208) 2024-09-13T17:44:15.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.454+0000 7ff08e0fa640 1 -- 172.21.15.102:0/3789687787 learned_addr learned my addr 172.21.15.102:0/3789687787 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:15.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.454+0000 7ff08e0fa640 1 -- 172.21.15.102:0/3789687787 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff088096a70 msgr2=0x7ff0880987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:15.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.454+0000 7ff08e0fa640 1 --2- 172.21.15.102:0/3789687787 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff088096a70 0x7ff0880987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:15.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.454+0000 7ff08e0fa640 1 -- 172.21.15.102:0/3789687787 >> v1:172.21.15.102:6789/0 conn(0x7ff088098d00 legacy=0x7ff08812c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:15.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.454+0000 7ff08e0fa640 1 -- 172.21.15.102:0/3789687787 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0880997a0 con 0x7ff088099110 2024-09-13T17:44:15.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.454+0000 7ff08d8f9640 1 --2- 172.21.15.102:0/3789687787 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff088096a70 0x7ff0880987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:15.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.455+0000 7ff08e0fa640 1 --2- 172.21.15.102:0/3789687787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff088099110 0x7ff08812c9f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff080009f20 tx=0x7ff080009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db123555a49a962 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:15.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.455+0000 7ff08c8f7640 1 -- 172.21.15.102:0/3789687787 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff080031400 con 0x7ff088099110 2024-09-13T17:44:15.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.455+0000 7ff08c8f7640 1 -- 172.21.15.102:0/3789687787 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08002d050 con 0x7ff088099110 2024-09-13T17:44:15.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.455+0000 7ff08c8f7640 1 -- 172.21.15.102:0/3789687787 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff080031790 con 0x7ff088099110 2024-09-13T17:44:15.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.455+0000 7ff08fb5d640 1 -- 172.21.15.102:0/3789687787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff088099110 msgr2=0x7ff08812c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:15.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.455+0000 7ff08fb5d640 1 --2- 172.21.15.102:0/3789687787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff088099110 0x7ff08812c9f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff080009f20 tx=0x7ff080009840 comp rx=0 tx=0).stop 2024-09-13T17:44:15.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.455+0000 7ff08fb5d640 1 -- 172.21.15.102:0/3789687787 shutdown_connections 2024-09-13T17:44:15.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.455+0000 7ff08fb5d640 1 --2- 172.21.15.102:0/3789687787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff088099110 0x7ff08812c9f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:15.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.455+0000 7ff08fb5d640 1 --2- 172.21.15.102:0/3789687787 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff088096a70 0x7ff0880987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:15.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.455+0000 7ff08fb5d640 1 -- 172.21.15.102:0/3789687787 >> 172.21.15.102:0/3789687787 conn(0x7ff08809e720 msgr2=0x7ff08809eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:15.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.455+0000 7ff08fb5d640 1 -- 172.21.15.102:0/3789687787 shutdown_connections 2024-09-13T17:44:15.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.455+0000 7ff08fb5d640 1 -- 172.21.15.102:0/3789687787 wait complete. 2024-09-13T17:44:15.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.456+0000 7ff08fb5d640 1 Processor -- start 2024-09-13T17:44:15.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.456+0000 7ff08fb5d640 1 -- start start 2024-09-13T17:44:15.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.457+0000 7ff08fb5d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff088096a70 0x7ff088128ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:15.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.457+0000 7ff08fb5d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff088098d00 0x7ff088129400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:15.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.457+0000 7ff08fb5d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff088099110 0x7ff088129940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:15.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.457+0000 7ff08fb5d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff088133370 con 0x7ff088096a70 2024-09-13T17:44:15.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.457+0000 7ff08fb5d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff08812a540 con 0x7ff088099110 2024-09-13T17:44:15.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.457+0000 7ff08fb5d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff08809a410 con 0x7ff088098d00 2024-09-13T17:44:15.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.457+0000 7ff08d8f9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff088096a70 0x7ff088128ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:15.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.457+0000 7ff08d8f9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff088096a70 0x7ff088128ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57044/0 (socket says 172.21.15.102:57044) 2024-09-13T17:44:15.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.457+0000 7ff08d8f9640 1 -- 172.21.15.102:0/579551883 learned_addr learned my addr 172.21.15.102:0/579551883 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:15.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.457+0000 7ff08e0fa640 1 --2- 172.21.15.102:0/579551883 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff088099110 0x7ff088129940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:15.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.457+0000 7ff08d0f8640 1 --2- 172.21.15.102:0/579551883 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff088098d00 0x7ff088129400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:15.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.457+0000 7ff08d8f9640 1 -- 172.21.15.102:0/579551883 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff088098d00 msgr2=0x7ff088129400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:15.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.457+0000 7ff08d8f9640 1 --2- 172.21.15.102:0/579551883 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff088098d00 0x7ff088129400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:15.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.457+0000 7ff08d8f9640 1 -- 172.21.15.102:0/579551883 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff088099110 msgr2=0x7ff088129940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:15.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.457+0000 7ff08d8f9640 1 --2- 172.21.15.102:0/579551883 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff088099110 0x7ff088129940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:15.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.457+0000 7ff08d8f9640 1 -- 172.21.15.102:0/579551883 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff084009390 con 0x7ff088096a70 2024-09-13T17:44:15.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.457+0000 7ff08d0f8640 1 --2- 172.21.15.102:0/579551883 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff088098d00 0x7ff088129400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:15.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.457+0000 7ff08e0fa640 1 --2- 172.21.15.102:0/579551883 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff088099110 0x7ff088129940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:15.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.457+0000 7ff08d8f9640 1 --2- 172.21.15.102:0/579551883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff088096a70 0x7ff088128ec0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff08400ec80 tx=0x7ff0840090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:15.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.457+0000 7ff07dffb640 1 -- 172.21.15.102:0/579551883 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08400eec0 con 0x7ff088096a70 2024-09-13T17:44:15.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.457+0000 7ff07dffb640 1 -- 172.21.15.102:0/579551883 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08400cbc0 con 0x7ff088096a70 2024-09-13T17:44:15.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.457+0000 7ff07dffb640 1 -- 172.21.15.102:0/579551883 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff084013470 con 0x7ff088096a70 2024-09-13T17:44:15.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.457+0000 7ff08fb5d640 1 -- 172.21.15.102:0/579551883 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff080009ae0 con 0x7ff088096a70 2024-09-13T17:44:15.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.458+0000 7ff08fb5d640 1 -- 172.21.15.102:0/579551883 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff088132cd0 con 0x7ff088096a70 2024-09-13T17:44:15.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.459+0000 7ff07dffb640 1 -- 172.21.15.102:0/579551883 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7ff0840137e0 con 0x7ff088096a70 2024-09-13T17:44:15.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.459+0000 7ff07dffb640 1 -- 172.21.15.102:0/579551883 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3660+0+0 (secure 0 0 0) 0x7ff0840423c0 con 0x7ff088096a70 2024-09-13T17:44:15.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.459+0000 7ff08fb5d640 1 -- 172.21.15.102:0/579551883 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff088131420 con 0x7ff088096a70 2024-09-13T17:44:15.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.463+0000 7ff07dffb640 1 -- 172.21.15.102:0/579551883 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7ff084021080 con 0x7ff088096a70 2024-09-13T17:44:15.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.613+0000 7ff08fb5d640 1 -- 172.21.15.102:0/579551883 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff088064000 con 0x7ff088096a70 2024-09-13T17:44:15.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.613+0000 7ff07dffb640 1 -- 172.21.15.102:0/579551883 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v32) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff084023ad0 con 0x7ff088096a70 2024-09-13T17:44:15.615 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-09-13T17:44:15.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.615+0000 7ff08fb5d640 1 -- 172.21.15.102:0/579551883 >> v1:172.21.15.102:6816/34937 conn(0x7ff06c031160 legacy=0x7ff06c033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:15.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.615+0000 7ff08fb5d640 1 -- 172.21.15.102:0/579551883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff088096a70 msgr2=0x7ff088128ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:15.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.615+0000 7ff08fb5d640 1 --2- 172.21.15.102:0/579551883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff088096a70 0x7ff088128ec0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff08400ec80 tx=0x7ff0840090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:15.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.615+0000 7ff08fb5d640 1 -- 172.21.15.102:0/579551883 shutdown_connections 2024-09-13T17:44:15.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.615+0000 7ff08fb5d640 1 --2- 172.21.15.102:0/579551883 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7ff088099110 0x7ff088129940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:15.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.615+0000 7ff08fb5d640 1 --2- 172.21.15.102:0/579551883 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7ff088098d00 0x7ff088129400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:15.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.615+0000 7ff08fb5d640 1 --2- 172.21.15.102:0/579551883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff088096a70 0x7ff088128ec0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:15.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.615+0000 7ff08fb5d640 1 -- 172.21.15.102:0/579551883 >> 172.21.15.102:0/579551883 conn(0x7ff08809e720 msgr2=0x7ff0880971f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:15.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.615+0000 7ff08fb5d640 1 -- 172.21.15.102:0/579551883 shutdown_connections 2024-09-13T17:44:15.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.615+0000 7ff08fb5d640 1 -- 172.21.15.102:0/579551883 wait complete. 2024-09-13T17:44:15.627 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-13T17:44:15.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.721+0000 7f05c7f5d640 1 Processor -- start 2024-09-13T17:44:15.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.721+0000 7f05c7f5d640 1 -- start start 2024-09-13T17:44:15.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.721+0000 7f05c7f5d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f05c0096df0 0x7f05c00971c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:15.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.721+0000 7f05c7f5d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f05c0097d30 0x7f05c012d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:15.724 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.721+0000 7f05c7f5d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05c006a850 con 0x7f05c0097700 2024-09-13T17:44:15.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.721+0000 7f05c7f5d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05c0098290 con 0x7f05c0097d30 2024-09-13T17:44:15.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.721+0000 7f05c7f5d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f05c0096bf0 con 0x7f05c0096df0 2024-09-13T17:44:15.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.722+0000 7f05c54f8640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f05c0097d30 0x7f05c012d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:15.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.722+0000 7f05c5cf9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f05c0096df0 0x7f05c00971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:15.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.722+0000 7f05c64fa640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f05c0097700 0x7f05c012d7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57212/0 (socket says 172.21.15.102:57212) 2024-09-13T17:44:15.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.722+0000 7f05c54f8640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f05c0097d30 0x7f05c012d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57046/0 (socket says 172.21.15.102:57046) 2024-09-13T17:44:15.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.722+0000 7f05c64fa640 1 -- 172.21.15.102:0/3569049565 learned_addr learned my addr 172.21.15.102:0/3569049565 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:44:15.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.722+0000 7f05c54f8640 1 -- 172.21.15.102:0/3569049565 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f05c0096df0 msgr2=0x7f05c00971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:15.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.722+0000 7f05c54f8640 1 --2- 172.21.15.102:0/3569049565 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f05c0096df0 0x7f05c00971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:15.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.722+0000 7f05c54f8640 1 -- 172.21.15.102:0/3569049565 >> v1:172.21.15.102:6789/0 conn(0x7f05c0097700 legacy=0x7f05c012d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:15.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.722+0000 7f05c54f8640 1 -- 172.21.15.102:0/3569049565 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c006a590 con 0x7f05c0097d30 2024-09-13T17:44:15.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.722+0000 7f05c5cf9640 1 --2- 172.21.15.102:0/3569049565 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f05c0096df0 0x7f05c00971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:15.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.722+0000 7f05c4cf7640 1 -- 172.21.15.102:0/3569049565 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 614902166 0 0) 0x7f05c006a850 con 0x7f05c0097700 2024-09-13T17:44:15.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.723+0000 7f05c54f8640 1 --2- 172.21.15.102:0/3569049565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05c0097d30 0x7f05c012d280 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009850 tx=0x7f05b00090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ebb76ae2ffcd79 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:15.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.723+0000 7f05c4cf7640 1 -- 172.21.15.102:0/3569049565 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b002ad00 con 0x7f05c0097d30 2024-09-13T17:44:15.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.723+0000 7f05c4cf7640 1 -- 172.21.15.102:0/3569049565 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b002ae60 con 0x7f05c0097d30 2024-09-13T17:44:15.726 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.723+0000 7f05c4cf7640 1 -- 172.21.15.102:0/3569049565 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b0032670 con 0x7f05c0097d30 2024-09-13T17:44:15.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.723+0000 7f05c7f5d640 1 -- 172.21.15.102:0/3569049565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05c0097d30 msgr2=0x7f05c012d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:15.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.723+0000 7f05c7f5d640 1 --2- 172.21.15.102:0/3569049565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05c0097d30 0x7f05c012d280 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009850 tx=0x7f05b00090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:15.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.723+0000 7f05c7f5d640 1 -- 172.21.15.102:0/3569049565 shutdown_connections 2024-09-13T17:44:15.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.723+0000 7f05c7f5d640 1 --2- 172.21.15.102:0/3569049565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05c0097d30 0x7f05c012d280 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:15.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.723+0000 7f05c7f5d640 1 --2- 172.21.15.102:0/3569049565 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f05c0096df0 0x7f05c00971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:15.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.723+0000 7f05c7f5d640 1 -- 172.21.15.102:0/3569049565 >> 172.21.15.102:0/3569049565 conn(0x7f05c009e720 msgr2=0x7f05c009eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:15.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.723+0000 7f05c7f5d640 1 -- 172.21.15.102:0/3569049565 shutdown_connections 2024-09-13T17:44:15.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.723+0000 7f05c7f5d640 1 -- 172.21.15.102:0/3569049565 wait complete. 2024-09-13T17:44:15.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.724+0000 7f05c7f5d640 1 Processor -- start 2024-09-13T17:44:15.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.724+0000 7f05c7f5d640 1 -- start start 2024-09-13T17:44:15.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.724+0000 7f05c7f5d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05c0096df0 0x7f05c013c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:15.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.725+0000 7f05c7f5d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05c0097700 0x7f05c013cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:15.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.725+0000 7f05c7f5d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f05c0097d30 0x7f05c013d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:15.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.725+0000 7f05c7f5d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c0069df0 con 0x7f05c0096df0 2024-09-13T17:44:15.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.725+0000 7f05c7f5d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05c0118a10 con 0x7f05c0097700 2024-09-13T17:44:15.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.725+0000 7f05c7f5d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f05c012b3f0 con 0x7f05c0097d30 2024-09-13T17:44:15.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.725+0000 7f05c5cf9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05c0096df0 0x7f05c013c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:15.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.725+0000 7f05c5cf9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05c0096df0 0x7f05c013c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57056/0 (socket says 172.21.15.102:57056) 2024-09-13T17:44:15.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.725+0000 7f05c5cf9640 1 -- 172.21.15.102:0/4108981632 learned_addr learned my addr 172.21.15.102:0/4108981632 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:15.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.725+0000 7f05c54f8640 1 --2- 172.21.15.102:0/4108981632 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05c0097700 0x7f05c013cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:15.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.725+0000 7f05c64fa640 1 --2- 172.21.15.102:0/4108981632 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f05c0097d30 0x7f05c013d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:15.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.725+0000 7f05c5cf9640 1 -- 172.21.15.102:0/4108981632 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f05c0097d30 msgr2=0x7f05c013d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:15.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.725+0000 7f05c5cf9640 1 --2- 172.21.15.102:0/4108981632 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f05c0097d30 0x7f05c013d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:15.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.725+0000 7f05c5cf9640 1 -- 172.21.15.102:0/4108981632 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05c0097700 msgr2=0x7f05c013cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:15.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.725+0000 7f05c5cf9640 1 --2- 172.21.15.102:0/4108981632 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05c0097700 0x7f05c013cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:15.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.725+0000 7f05c5cf9640 1 -- 172.21.15.102:0/4108981632 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05bc009390 con 0x7f05c0096df0 2024-09-13T17:44:15.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.725+0000 7f05c5cf9640 1 --2- 172.21.15.102:0/4108981632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05c0096df0 0x7f05c013c720 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f05bc012eb0 tx=0x7f05bc0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:15.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.725+0000 7f05b5ffb640 1 -- 172.21.15.102:0/4108981632 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc00cea0 con 0x7f05c0096df0 2024-09-13T17:44:15.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.725+0000 7f05b5ffb640 1 -- 172.21.15.102:0/4108981632 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05bc01a430 con 0x7f05c0096df0 2024-09-13T17:44:15.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.725+0000 7f05b5ffb640 1 -- 172.21.15.102:0/4108981632 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc01a7d0 con 0x7f05c0096df0 2024-09-13T17:44:15.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.725+0000 7f05c7f5d640 1 -- 172.21.15.102:0/4108981632 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b0009390 con 0x7f05c0096df0 2024-09-13T17:44:15.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.725+0000 7f05c7f5d640 1 -- 172.21.15.102:0/4108981632 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05c0119af0 con 0x7f05c0096df0 2024-09-13T17:44:15.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.726+0000 7f05b5ffb640 1 -- 172.21.15.102:0/4108981632 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f05bc01ab40 con 0x7f05c0096df0 2024-09-13T17:44:15.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.726+0000 7f05b5ffb640 1 -- 172.21.15.102:0/4108981632 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3660+0+0 (secure 0 0 0) 0x7f05bc045e60 con 0x7f05c0096df0 2024-09-13T17:44:15.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.726+0000 7f05c7f5d640 1 -- 172.21.15.102:0/4108981632 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05c0064000 con 0x7f05c0096df0 2024-09-13T17:44:15.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.731+0000 7f05b5ffb640 1 -- 172.21.15.102:0/4108981632 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f05bc01f080 con 0x7f05c0096df0 2024-09-13T17:44:15.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.880+0000 7f05c7f5d640 1 -- 172.21.15.102:0/4108981632 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f05c00676b0 con 0x7f05c0096df0 2024-09-13T17:44:15.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.881+0000 7f05b5ffb640 1 -- 172.21.15.102:0/4108981632 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v32) v1 ==== 92+0+10 (secure 0 0 0) 0x7f05bc0233b0 con 0x7f05c0096df0 2024-09-13T17:44:15.883 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-09-13T17:44:15.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.883+0000 7f05c7f5d640 1 -- 172.21.15.102:0/4108981632 >> v1:172.21.15.102:6816/34937 conn(0x7f05a4031160 legacy=0x7f05a4033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:15.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.883+0000 7f05c7f5d640 1 -- 172.21.15.102:0/4108981632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05c0096df0 msgr2=0x7f05c013c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:15.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.883+0000 7f05c7f5d640 1 --2- 172.21.15.102:0/4108981632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05c0096df0 0x7f05c013c720 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f05bc012eb0 tx=0x7f05bc0090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:15.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.883+0000 7f05c7f5d640 1 -- 172.21.15.102:0/4108981632 shutdown_connections 2024-09-13T17:44:15.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.883+0000 7f05c7f5d640 1 --2- 172.21.15.102:0/4108981632 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f05c0097d30 0x7f05c013d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:15.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.883+0000 7f05c7f5d640 1 --2- 172.21.15.102:0/4108981632 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05c0097700 0x7f05c013cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:15.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.883+0000 7f05c7f5d640 1 --2- 172.21.15.102:0/4108981632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05c0096df0 0x7f05c013c720 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:15.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.883+0000 7f05c7f5d640 1 -- 172.21.15.102:0/4108981632 >> 172.21.15.102:0/4108981632 conn(0x7f05c009e720 msgr2=0x7f05c009d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:15.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.883+0000 7f05c7f5d640 1 -- 172.21.15.102:0/4108981632 shutdown_connections 2024-09-13T17:44:15.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.883+0000 7f05c7f5d640 1 -- 172.21.15.102:0/4108981632 wait complete. 2024-09-13T17:44:15.895 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:44:15.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.988+0000 7fe9d8446640 1 Processor -- start 2024-09-13T17:44:15.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.988+0000 7fe9d8446640 1 -- start start 2024-09-13T17:44:15.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.989+0000 7fe9d8446640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe9d0097340 0x7fe9d0132510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:15.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.989+0000 7fe9d8446640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9d0097740 0x7fe9d0134a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:15.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.989+0000 7fe9d8446640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d0098040 con 0x7fe9d009a620 2024-09-13T17:44:15.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.989+0000 7fe9d8446640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9d009a9f0 con 0x7fe9d0097340 2024-09-13T17:44:15.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.989+0000 7fe9d8446640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d00982a0 con 0x7fe9d0097740 2024-09-13T17:44:15.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.989+0000 7fe9d69e3640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9d0097740 0x7fe9d0134a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:15.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.989+0000 7fe9d59e1640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe9d0097340 0x7fe9d0132510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:15.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.989+0000 7fe9d61e2640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fe9d009a620 0x7fe9d0096c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57222/0 (socket says 172.21.15.102:57222) 2024-09-13T17:44:15.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.989+0000 7fe9d69e3640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9d0097740 0x7fe9d0134a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:57830/0 (socket says 172.21.15.102:57830) 2024-09-13T17:44:15.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.989+0000 7fe9d59e1640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe9d0097340 0x7fe9d0132510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57066/0 (socket says 172.21.15.102:57066) 2024-09-13T17:44:15.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.989+0000 7fe9d61e2640 1 -- 172.21.15.102:0/2456812685 learned_addr learned my addr 172.21.15.102:0/2456812685 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:44:15.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.989+0000 7fe9d59e1640 1 -- 172.21.15.102:0/2456812685 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9d0097740 msgr2=0x7fe9d0134a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:15.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.989+0000 7fe9d59e1640 1 --2- 172.21.15.102:0/2456812685 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9d0097740 0x7fe9d0134a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:15.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.989+0000 7fe9d59e1640 1 -- 172.21.15.102:0/2456812685 >> v1:172.21.15.102:6789/0 conn(0x7fe9d009a620 legacy=0x7fe9d0096c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:15.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.989+0000 7fe9d51e0640 1 -- 172.21.15.102:0/2456812685 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1949077573 0 0) 0x7fe9d0098040 con 0x7fe9d009a620 2024-09-13T17:44:15.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.989+0000 7fe9d59e1640 1 -- 172.21.15.102:0/2456812685 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d009ad50 con 0x7fe9d0097340 2024-09-13T17:44:16.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.990+0000 7fe9d59e1640 1 --2- 172.21.15.102:0/2456812685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9d0097340 0x7fe9d0132510 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009850 tx=0x7fe9cc0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33534be775ec1f5d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:16.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.990+0000 7fe9d51e0640 1 -- 172.21.15.102:0/2456812685 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc02ad00 con 0x7fe9d0097340 2024-09-13T17:44:16.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.990+0000 7fe9d51e0640 1 -- 172.21.15.102:0/2456812685 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9cc02ae60 con 0x7fe9d0097340 2024-09-13T17:44:16.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.990+0000 7fe9d51e0640 1 -- 172.21.15.102:0/2456812685 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc032670 con 0x7fe9d0097340 2024-09-13T17:44:16.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.991+0000 7fe9d8446640 1 -- 172.21.15.102:0/2456812685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9d0097340 msgr2=0x7fe9d0132510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:16.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.991+0000 7fe9d8446640 1 --2- 172.21.15.102:0/2456812685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9d0097340 0x7fe9d0132510 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009850 tx=0x7fe9cc0090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:16.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.991+0000 7fe9d8446640 1 -- 172.21.15.102:0/2456812685 shutdown_connections 2024-09-13T17:44:16.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.991+0000 7fe9d8446640 1 --2- 172.21.15.102:0/2456812685 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9d0097740 0x7fe9d0134a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:16.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.991+0000 7fe9d8446640 1 --2- 172.21.15.102:0/2456812685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9d0097340 0x7fe9d0132510 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:16.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.991+0000 7fe9d8446640 1 -- 172.21.15.102:0/2456812685 >> 172.21.15.102:0/2456812685 conn(0x7fe9d009e6b0 msgr2=0x7fe9d009eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:16.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.991+0000 7fe9d8446640 1 -- 172.21.15.102:0/2456812685 shutdown_connections 2024-09-13T17:44:16.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.991+0000 7fe9d8446640 1 -- 172.21.15.102:0/2456812685 wait complete. 2024-09-13T17:44:16.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.992+0000 7fe9d8446640 1 Processor -- start 2024-09-13T17:44:16.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.992+0000 7fe9d8446640 1 -- start start 2024-09-13T17:44:16.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.992+0000 7fe9d8446640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9d0097340 0x7fe9d012c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:16.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.992+0000 7fe9d8446640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9d0097740 0x7fe9d0119740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:16.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.992+0000 7fe9d8446640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe9d009a620 0x7fe9d0119c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:16.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.992+0000 7fe9d8446640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d009ce20 con 0x7fe9d0097340 2024-09-13T17:44:16.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.992+0000 7fe9d8446640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d0069d20 con 0x7fe9d009a620 2024-09-13T17:44:16.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.992+0000 7fe9d8446640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d011ce40 con 0x7fe9d0097740 2024-09-13T17:44:16.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.992+0000 7fe9d59e1640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9d0097740 0x7fe9d0119740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:16.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.992+0000 7fe9d61e2640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9d0097340 0x7fe9d012c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:16.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.992+0000 7fe9d69e3640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe9d009a620 0x7fe9d0119c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:16.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.992+0000 7fe9d59e1640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9d0097740 0x7fe9d0119740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:57842/0 (socket says 172.21.15.102:57842) 2024-09-13T17:44:16.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.992+0000 7fe9d59e1640 1 -- 172.21.15.102:0/3959519973 learned_addr learned my addr 172.21.15.102:0/3959519973 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:16.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.993+0000 7fe9d59e1640 1 -- 172.21.15.102:0/3959519973 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe9d009a620 msgr2=0x7fe9d0119c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:16.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.993+0000 7fe9d59e1640 1 --2- 172.21.15.102:0/3959519973 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe9d009a620 0x7fe9d0119c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:16.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.993+0000 7fe9d59e1640 1 -- 172.21.15.102:0/3959519973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9d0097340 msgr2=0x7fe9d012c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:16.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.993+0000 7fe9d59e1640 1 --2- 172.21.15.102:0/3959519973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9d0097340 0x7fe9d012c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:16.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.993+0000 7fe9d59e1640 1 -- 172.21.15.102:0/3959519973 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9cc009390 con 0x7fe9d0097740 2024-09-13T17:44:16.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.993+0000 7fe9d61e2640 1 --2- 172.21.15.102:0/3959519973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9d0097340 0x7fe9d012c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:16.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.993+0000 7fe9d59e1640 1 --2- 172.21.15.102:0/3959519973 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9d0097740 0x7fe9d0119740 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009850 tx=0x7fe9cc032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:16.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.993+0000 7fe9c27fc640 1 -- 172.21.15.102:0/3959519973 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc031700 con 0x7fe9d0097740 2024-09-13T17:44:16.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.993+0000 7fe9c27fc640 1 -- 172.21.15.102:0/3959519973 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9cc031860 con 0x7fe9d0097740 2024-09-13T17:44:16.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.993+0000 7fe9d8446640 1 -- 172.21.15.102:0/3959519973 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9d011b280 con 0x7fe9d0097740 2024-09-13T17:44:16.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.993+0000 7fe9c27fc640 1 -- 172.21.15.102:0/3959519973 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc031c20 con 0x7fe9d0097740 2024-09-13T17:44:16.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.993+0000 7fe9d8446640 1 -- 172.21.15.102:0/3959519973 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d012cc20 con 0x7fe9d0097740 2024-09-13T17:44:16.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.994+0000 7fe9c27fc640 1 -- 172.21.15.102:0/3959519973 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fe9cc039040 con 0x7fe9d0097740 2024-09-13T17:44:16.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.994+0000 7fe9c27fc640 1 -- 172.21.15.102:0/3959519973 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3660+0+0 (secure 0 0 0) 0x7fe9cc062270 con 0x7fe9d0097740 2024-09-13T17:44:16.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.994+0000 7fe9d8446640 1 -- 172.21.15.102:0/3959519973 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9d0064000 con 0x7fe9d0097740 2024-09-13T17:44:16.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:15.998+0000 7fe9c27fc640 1 -- 172.21.15.102:0/3959519973 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fe9cc037080 con 0x7fe9d0097740 2024-09-13T17:44:16.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.154+0000 7fe9d8446640 1 -- 172.21.15.102:0/3959519973 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe9d006e5e0 con 0x7fe9d0097740 2024-09-13T17:44:16.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.155+0000 7fe9c27fc640 1 -- 172.21.15.102:0/3959519973 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v32) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe9cc049020 con 0x7fe9d0097740 2024-09-13T17:44:16.157 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-09-13T17:44:16.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.157+0000 7fe9d8446640 1 -- 172.21.15.102:0/3959519973 >> v1:172.21.15.102:6816/34937 conn(0x7fe9b4031160 legacy=0x7fe9b4033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:16.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.157+0000 7fe9d8446640 1 -- 172.21.15.102:0/3959519973 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9d0097740 msgr2=0x7fe9d0119740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:16.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.157+0000 7fe9d8446640 1 --2- 172.21.15.102:0/3959519973 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9d0097740 0x7fe9d0119740 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009850 tx=0x7fe9cc032000 comp rx=0 tx=0).stop 2024-09-13T17:44:16.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.157+0000 7fe9d8446640 1 -- 172.21.15.102:0/3959519973 shutdown_connections 2024-09-13T17:44:16.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.157+0000 7fe9d8446640 1 --2- 172.21.15.102:0/3959519973 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe9d009a620 0x7fe9d0119c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:16.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.157+0000 7fe9d8446640 1 --2- 172.21.15.102:0/3959519973 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe9d0097740 0x7fe9d0119740 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:16.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.157+0000 7fe9d8446640 1 --2- 172.21.15.102:0/3959519973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9d0097340 0x7fe9d012c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:16.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.157+0000 7fe9d8446640 1 -- 172.21.15.102:0/3959519973 >> 172.21.15.102:0/3959519973 conn(0x7fe9d009e6b0 msgr2=0x7fe9d012d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:16.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.157+0000 7fe9d8446640 1 -- 172.21.15.102:0/3959519973 shutdown_connections 2024-09-13T17:44:16.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.157+0000 7fe9d8446640 1 -- 172.21.15.102:0/3959519973 wait complete. 2024-09-13T17:44:16.169 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:44:16.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.264+0000 7f97fb046640 1 Processor -- start 2024-09-13T17:44:16.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.264+0000 7f97fb046640 1 -- start start 2024-09-13T17:44:16.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.264+0000 7f97fb046640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f97f4098ed0 0x7f97f4131610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:16.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.264+0000 7f97fb046640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f97f4131b50 0x7f97f4133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:16.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.264+0000 7f97fb046640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f40999e0 con 0x7f97f4096a70 2024-09-13T17:44:16.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.264+0000 7f97fb046640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f4069c30 con 0x7f97f4098ed0 2024-09-13T17:44:16.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.264+0000 7f97fb046640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f97f406ed00 con 0x7f97f4131b50 2024-09-13T17:44:16.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.265+0000 7f97f3fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f97f4098ed0 0x7f97f4131610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:16.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.265+0000 7f97f95e3640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f97f4131b50 0x7f97f4133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:16.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.265+0000 7f97f8de2640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f97f4096a70 0x7f97f40987c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57226/0 (socket says 172.21.15.102:57226) 2024-09-13T17:44:16.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.265+0000 7f97f8de2640 1 -- 172.21.15.102:0/3731092416 learned_addr learned my addr 172.21.15.102:0/3731092416 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:44:16.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.265+0000 7f97f37fe640 1 -- 172.21.15.102:0/3731092416 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2682717546 0 0) 0x7f97f40999e0 con 0x7f97f4096a70 2024-09-13T17:44:16.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.265+0000 7f97f37fe640 1 -- 172.21.15.102:0/3731092416 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97e40034e0 con 0x7f97f4096a70 2024-09-13T17:44:16.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.265+0000 7f97f3fff640 1 -- 172.21.15.102:0/3731092416 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f97f4131b50 msgr2=0x7f97f4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:16.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.265+0000 7f97f3fff640 1 --2- 172.21.15.102:0/3731092416 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f97f4131b50 0x7f97f4133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:16.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.265+0000 7f97f3fff640 1 -- 172.21.15.102:0/3731092416 >> v1:172.21.15.102:6789/0 conn(0x7f97f4096a70 legacy=0x7f97f40987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:16.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.265+0000 7f97f3fff640 1 -- 172.21.15.102:0/3731092416 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f40997a0 con 0x7f97f4098ed0 2024-09-13T17:44:16.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.265+0000 7f97f95e3640 1 --2- 172.21.15.102:0/3731092416 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f97f4131b50 0x7f97f4133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:16.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.266+0000 7f97f3fff640 1 --2- 172.21.15.102:0/3731092416 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f97f4098ed0 0x7f97f4131610 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f97e00098b0 tx=0x7f97e00090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a37dfc833aa4bbc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:16.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.266+0000 7f97f37fe640 1 -- 172.21.15.102:0/3731092416 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e002acc0 con 0x7f97f4098ed0 2024-09-13T17:44:16.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.266+0000 7f97f37fe640 1 -- 172.21.15.102:0/3731092416 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e002ae20 con 0x7f97f4098ed0 2024-09-13T17:44:16.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.266+0000 7f97f37fe640 1 -- 172.21.15.102:0/3731092416 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e0032670 con 0x7f97f4098ed0 2024-09-13T17:44:16.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.266+0000 7f97fb046640 1 -- 172.21.15.102:0/3731092416 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f97f4098ed0 msgr2=0x7f97f4131610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:16.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.266+0000 7f97fb046640 1 --2- 172.21.15.102:0/3731092416 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f97f4098ed0 0x7f97f4131610 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f97e00098b0 tx=0x7f97e00090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:16.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.266+0000 7f97fb046640 1 -- 172.21.15.102:0/3731092416 shutdown_connections 2024-09-13T17:44:16.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.266+0000 7f97fb046640 1 --2- 172.21.15.102:0/3731092416 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f97f4131b50 0x7f97f4133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:16.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.266+0000 7f97fb046640 1 --2- 172.21.15.102:0/3731092416 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f97f4098ed0 0x7f97f4131610 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:16.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.266+0000 7f97fb046640 1 -- 172.21.15.102:0/3731092416 >> 172.21.15.102:0/3731092416 conn(0x7f97f409e720 msgr2=0x7f97f409eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:16.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.266+0000 7f97fb046640 1 -- 172.21.15.102:0/3731092416 shutdown_connections 2024-09-13T17:44:16.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.266+0000 7f97fb046640 1 -- 172.21.15.102:0/3731092416 wait complete. 2024-09-13T17:44:16.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.267+0000 7f97fb046640 1 Processor -- start 2024-09-13T17:44:16.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.267+0000 7f97fb046640 1 -- start start 2024-09-13T17:44:16.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.268+0000 7f97fb046640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f97f4096a70 0x7f97f4114e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:16.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.268+0000 7f97fb046640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97f4098ed0 0x7f97f4115360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:16.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.268+0000 7f97fb046640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f97f4131b50 0x7f97f41158a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:16.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.268+0000 7f97fb046640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f4097d80 con 0x7f97f4098ed0 2024-09-13T17:44:16.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.268+0000 7f97fb046640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f4097f70 con 0x7f97f4096a70 2024-09-13T17:44:16.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.268+0000 7f97fb046640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f97f409a410 con 0x7f97f4131b50 2024-09-13T17:44:16.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.268+0000 7f97f8de2640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f97f4096a70 0x7f97f4114e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:16.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.268+0000 7f97f3fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97f4098ed0 0x7f97f4115360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:16.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.268+0000 7f97f8de2640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f97f4096a70 0x7f97f4114e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35318/0 (socket says 172.21.15.102:35318) 2024-09-13T17:44:16.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.268+0000 7f97f8de2640 1 -- 172.21.15.102:0/3907274349 learned_addr learned my addr 172.21.15.102:0/3907274349 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:16.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.268+0000 7f97f95e3640 1 --2- 172.21.15.102:0/3907274349 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f97f4131b50 0x7f97f41158a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:16.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.268+0000 7f97f3fff640 1 -- 172.21.15.102:0/3907274349 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f97f4131b50 msgr2=0x7f97f41158a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:16.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.268+0000 7f97f3fff640 1 --2- 172.21.15.102:0/3907274349 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f97f4131b50 0x7f97f41158a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:16.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.268+0000 7f97f3fff640 1 -- 172.21.15.102:0/3907274349 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f97f4096a70 msgr2=0x7f97f4114e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:16.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.268+0000 7f97f3fff640 1 --2- 172.21.15.102:0/3907274349 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f97f4096a70 0x7f97f4114e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:16.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.268+0000 7f97f3fff640 1 -- 172.21.15.102:0/3907274349 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ec009a20 con 0x7f97f4098ed0 2024-09-13T17:44:16.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.268+0000 7f97f8de2640 1 --2- 172.21.15.102:0/3907274349 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f97f4096a70 0x7f97f4114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:16.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.268+0000 7f97f3fff640 1 --2- 172.21.15.102:0/3907274349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97f4098ed0 0x7f97f4115360 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f97e0031000 tx=0x7f97e0032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:16.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.269+0000 7f97f0ff9640 1 -- 172.21.15.102:0/3907274349 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e003d700 con 0x7f97f4098ed0 2024-09-13T17:44:16.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.269+0000 7f97f0ff9640 1 -- 172.21.15.102:0/3907274349 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e003d8a0 con 0x7f97f4098ed0 2024-09-13T17:44:16.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.269+0000 7f97fb046640 1 -- 172.21.15.102:0/3907274349 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97e0009390 con 0x7f97f4098ed0 2024-09-13T17:44:16.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.269+0000 7f97f0ff9640 1 -- 172.21.15.102:0/3907274349 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e003dc60 con 0x7f97f4098ed0 2024-09-13T17:44:16.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.269+0000 7f97fb046640 1 -- 172.21.15.102:0/3907274349 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f4131630 con 0x7f97f4098ed0 2024-09-13T17:44:16.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.269+0000 7f97f0ff9640 1 -- 172.21.15.102:0/3907274349 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f97e0039040 con 0x7f97f4098ed0 2024-09-13T17:44:16.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.270+0000 7f97f0ff9640 1 -- 172.21.15.102:0/3907274349 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3660+0+0 (secure 0 0 0) 0x7f97e0061a10 con 0x7f97f4098ed0 2024-09-13T17:44:16.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.270+0000 7f97fb046640 1 -- 172.21.15.102:0/3907274349 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f4064000 con 0x7f97f4098ed0 2024-09-13T17:44:16.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.274+0000 7f97f0ff9640 1 -- 172.21.15.102:0/3907274349 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f97e0042070 con 0x7f97f4098ed0 2024-09-13T17:44:16.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.427+0000 7f97fb046640 1 -- 172.21.15.102:0/3907274349 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f97f4116dc0 con 0x7f97f4098ed0 2024-09-13T17:44:16.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.427+0000 7f97f0ff9640 1 -- 172.21.15.102:0/3907274349 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v32) v1 ==== 100+0+11 (secure 0 0 0) 0x7f97e0042030 con 0x7f97f4098ed0 2024-09-13T17:44:16.429 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-09-13T17:44:16.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.429+0000 7f97fb046640 1 -- 172.21.15.102:0/3907274349 >> v1:172.21.15.102:6816/34937 conn(0x7f97d8031160 legacy=0x7f97d8033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:16.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.429+0000 7f97fb046640 1 -- 172.21.15.102:0/3907274349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97f4098ed0 msgr2=0x7f97f4115360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:16.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.429+0000 7f97fb046640 1 --2- 172.21.15.102:0/3907274349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97f4098ed0 0x7f97f4115360 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f97e0031000 tx=0x7f97e0032000 comp rx=0 tx=0).stop 2024-09-13T17:44:16.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.429+0000 7f97fb046640 1 -- 172.21.15.102:0/3907274349 shutdown_connections 2024-09-13T17:44:16.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.429+0000 7f97fb046640 1 --2- 172.21.15.102:0/3907274349 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f97f4131b50 0x7f97f41158a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:16.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.429+0000 7f97fb046640 1 --2- 172.21.15.102:0/3907274349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97f4098ed0 0x7f97f4115360 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:16.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.429+0000 7f97fb046640 1 --2- 172.21.15.102:0/3907274349 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f97f4096a70 0x7f97f4114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:16.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.429+0000 7f97fb046640 1 -- 172.21.15.102:0/3907274349 >> 172.21.15.102:0/3907274349 conn(0x7f97f409e720 msgr2=0x7f97f409dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:16.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.429+0000 7f97fb046640 1 -- 172.21.15.102:0/3907274349 shutdown_connections 2024-09-13T17:44:16.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.429+0000 7f97fb046640 1 -- 172.21.15.102:0/3907274349 wait complete. 2024-09-13T17:44:16.442 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:44:16.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.533+0000 7f4e274af640 1 Processor -- start 2024-09-13T17:44:16.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.533+0000 7f4e274af640 1 -- start start 2024-09-13T17:44:16.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.534+0000 7f4e274af640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e2009abb0 0x7f4e2012c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:16.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.534+0000 7f4e274af640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4e2012cab0 0x7f4e20097490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:16.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.534+0000 7f4e274af640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e2006a530 con 0x7f4e2012c6a0 2024-09-13T17:44:16.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.534+0000 7f4e274af640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2009af80 con 0x7f4e2009abb0 2024-09-13T17:44:16.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.534+0000 7f4e274af640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4e20098be0 con 0x7f4e2012cab0 2024-09-13T17:44:16.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.534+0000 7f4e2524b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e2009abb0 0x7f4e2012c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:16.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.534+0000 7f4e25a4c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4e2012cab0 0x7f4e20097490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:16.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.534+0000 7f4e24a4a640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f4e2012c6a0 0x7f4e20096d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57230/0 (socket says 172.21.15.102:57230) 2024-09-13T17:44:16.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.534+0000 7f4e24a4a640 1 -- 172.21.15.102:0/2367855831 learned_addr learned my addr 172.21.15.102:0/2367855831 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:44:16.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.534+0000 7f4e2524b640 1 -- 172.21.15.102:0/2367855831 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4e2012cab0 msgr2=0x7f4e20097490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:16.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.534+0000 7f4e2524b640 1 --2- 172.21.15.102:0/2367855831 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4e2012cab0 0x7f4e20097490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:16.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.534+0000 7f4e17fff640 1 -- 172.21.15.102:0/2367855831 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1194439964 0 0) 0x7f4e2006a530 con 0x7f4e2012c6a0 2024-09-13T17:44:16.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.534+0000 7f4e2524b640 1 -- 172.21.15.102:0/2367855831 >> v1:172.21.15.102:6789/0 conn(0x7f4e2012c6a0 legacy=0x7f4e20096d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:16.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.534+0000 7f4e2524b640 1 -- 172.21.15.102:0/2367855831 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e20098930 con 0x7f4e2009abb0 2024-09-13T17:44:16.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.534+0000 7f4e25a4c640 1 --2- 172.21.15.102:0/2367855831 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4e2012cab0 0x7f4e20097490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:16.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.535+0000 7f4e2524b640 1 --2- 172.21.15.102:0/2367855831 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e2009abb0 0x7f4e2012c160 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c009880 tx=0x7f4e1c0090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=153a4f2f796089a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:16.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.535+0000 7f4e17fff640 1 -- 172.21.15.102:0/2367855831 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c02ac50 con 0x7f4e2009abb0 2024-09-13T17:44:16.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.535+0000 7f4e17fff640 1 -- 172.21.15.102:0/2367855831 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e1c02adb0 con 0x7f4e2009abb0 2024-09-13T17:44:16.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.535+0000 7f4e17fff640 1 -- 172.21.15.102:0/2367855831 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c032670 con 0x7f4e2009abb0 2024-09-13T17:44:16.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.536+0000 7f4e274af640 1 -- 172.21.15.102:0/2367855831 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e2009abb0 msgr2=0x7f4e2012c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:16.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.536+0000 7f4e274af640 1 --2- 172.21.15.102:0/2367855831 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e2009abb0 0x7f4e2012c160 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c009880 tx=0x7f4e1c0090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:16.541 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.536+0000 7f4e274af640 1 -- 172.21.15.102:0/2367855831 shutdown_connections 2024-09-13T17:44:16.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.536+0000 7f4e274af640 1 --2- 172.21.15.102:0/2367855831 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4e2012cab0 0x7f4e20097490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:16.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.536+0000 7f4e274af640 1 --2- 172.21.15.102:0/2367855831 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e2009abb0 0x7f4e2012c160 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:16.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.536+0000 7f4e274af640 1 -- 172.21.15.102:0/2367855831 >> 172.21.15.102:0/2367855831 conn(0x7f4e2009e820 msgr2=0x7f4e2009ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:16.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.536+0000 7f4e274af640 1 -- 172.21.15.102:0/2367855831 shutdown_connections 2024-09-13T17:44:16.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.536+0000 7f4e274af640 1 -- 172.21.15.102:0/2367855831 wait complete. 2024-09-13T17:44:16.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.536+0000 7f4e274af640 1 Processor -- start 2024-09-13T17:44:16.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.536+0000 7f4e274af640 1 -- start start 2024-09-13T17:44:16.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.537+0000 7f4e274af640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e2009abb0 0x7f4e20115be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:16.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.537+0000 7f4e274af640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e2012c6a0 0x7f4e20116120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:16.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.537+0000 7f4e274af640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4e2012cab0 0x7f4e2013c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:16.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.537+0000 7f4e274af640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2012b6d0 con 0x7f4e2012c6a0 2024-09-13T17:44:16.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.537+0000 7f4e274af640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e200981a0 con 0x7f4e2009abb0 2024-09-13T17:44:16.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.537+0000 7f4e274af640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4e20118bc0 con 0x7f4e2012cab0 2024-09-13T17:44:16.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.537+0000 7f4e2524b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e2009abb0 0x7f4e20115be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:16.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.537+0000 7f4e24a4a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e2012c6a0 0x7f4e20116120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:16.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.537+0000 7f4e2524b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e2009abb0 0x7f4e20115be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35326/0 (socket says 172.21.15.102:35326) 2024-09-13T17:44:16.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.537+0000 7f4e24a4a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e2012c6a0 0x7f4e20116120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57094/0 (socket says 172.21.15.102:57094) 2024-09-13T17:44:16.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.537+0000 7f4e2524b640 1 -- 172.21.15.102:0/1203688678 learned_addr learned my addr 172.21.15.102:0/1203688678 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:16.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.537+0000 7f4e25a4c640 1 --2- 172.21.15.102:0/1203688678 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4e2012cab0 0x7f4e2013c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:16.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.537+0000 7f4e2524b640 1 -- 172.21.15.102:0/1203688678 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4e2012cab0 msgr2=0x7f4e2013c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:16.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.537+0000 7f4e2524b640 1 --2- 172.21.15.102:0/1203688678 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4e2012cab0 0x7f4e2013c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:16.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.537+0000 7f4e2524b640 1 -- 172.21.15.102:0/1203688678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e2012c6a0 msgr2=0x7f4e20116120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:16.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.537+0000 7f4e2524b640 1 --2- 172.21.15.102:0/1203688678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e2012c6a0 0x7f4e20116120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:16.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.537+0000 7f4e2524b640 1 -- 172.21.15.102:0/1203688678 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e18009a20 con 0x7f4e2009abb0 2024-09-13T17:44:16.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.537+0000 7f4e24a4a640 1 --2- 172.21.15.102:0/1203688678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e2012c6a0 0x7f4e20116120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:16.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.538+0000 7f4e2524b640 1 --2- 172.21.15.102:0/1203688678 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e2009abb0 0x7f4e20115be0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c0098e0 tx=0x7f4e1c032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:16.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.538+0000 7f4e157fa640 1 -- 172.21.15.102:0/1203688678 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c031780 con 0x7f4e2009abb0 2024-09-13T17:44:16.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.538+0000 7f4e157fa640 1 -- 172.21.15.102:0/1203688678 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e1c0318e0 con 0x7f4e2009abb0 2024-09-13T17:44:16.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.538+0000 7f4e157fa640 1 -- 172.21.15.102:0/1203688678 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c031ca0 con 0x7f4e2009abb0 2024-09-13T17:44:16.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.538+0000 7f4e274af640 1 -- 172.21.15.102:0/1203688678 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e1c009390 con 0x7f4e2009abb0 2024-09-13T17:44:16.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.538+0000 7f4e274af640 1 -- 172.21.15.102:0/1203688678 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e201195f0 con 0x7f4e2009abb0 2024-09-13T17:44:16.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.538+0000 7f4e157fa640 1 -- 172.21.15.102:0/1203688678 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f4e1c03c400 con 0x7f4e2009abb0 2024-09-13T17:44:16.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.539+0000 7f4e157fa640 1 -- 172.21.15.102:0/1203688678 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3660+0+0 (secure 0 0 0) 0x7f4e1c062120 con 0x7f4e2009abb0 2024-09-13T17:44:16.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.539+0000 7f4e274af640 1 -- 172.21.15.102:0/1203688678 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e20098e00 con 0x7f4e2009abb0 2024-09-13T17:44:16.545 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.543+0000 7f4e157fa640 1 -- 172.21.15.102:0/1203688678 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f4e1c03c6b0 con 0x7f4e2009abb0 2024-09-13T17:44:16.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.706+0000 7f4e274af640 1 -- 172.21.15.102:0/1203688678 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4e20064000 con 0x7f4e2009abb0 2024-09-13T17:44:16.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.706+0000 7f4e157fa640 1 -- 172.21.15.102:0/1203688678 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v32) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4e1c037070 con 0x7f4e2009abb0 2024-09-13T17:44:16.708 INFO:teuthology.orchestra.run.smithi102.stderr:listed 0 entries 2024-09-13T17:44:16.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.708+0000 7f4e274af640 1 -- 172.21.15.102:0/1203688678 >> v1:172.21.15.102:6816/34937 conn(0x7f4e04031160 legacy=0x7f4e04033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:16.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.708+0000 7f4e274af640 1 -- 172.21.15.102:0/1203688678 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e2009abb0 msgr2=0x7f4e20115be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:16.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.708+0000 7f4e274af640 1 --2- 172.21.15.102:0/1203688678 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e2009abb0 0x7f4e20115be0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c0098e0 tx=0x7f4e1c032000 comp rx=0 tx=0).stop 2024-09-13T17:44:16.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.708+0000 7f4e274af640 1 -- 172.21.15.102:0/1203688678 shutdown_connections 2024-09-13T17:44:16.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.708+0000 7f4e274af640 1 --2- 172.21.15.102:0/1203688678 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f4e2012cab0 0x7f4e2013c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:16.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.708+0000 7f4e274af640 1 --2- 172.21.15.102:0/1203688678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e2012c6a0 0x7f4e20116120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:16.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.708+0000 7f4e274af640 1 --2- 172.21.15.102:0/1203688678 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f4e2009abb0 0x7f4e20115be0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:16.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.708+0000 7f4e274af640 1 -- 172.21.15.102:0/1203688678 >> 172.21.15.102:0/1203688678 conn(0x7f4e2009e820 msgr2=0x7f4e2009d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:16.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.708+0000 7f4e274af640 1 -- 172.21.15.102:0/1203688678 shutdown_connections 2024-09-13T17:44:16.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.708+0000 7f4e274af640 1 -- 172.21.15.102:0/1203688678 wait complete. 2024-09-13T17:44:16.720 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi171.front.sepia.ceph.com... 2024-09-13T17:44:16.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:16.720 DEBUG:teuthology.orchestra.run.smithi171:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-13T17:44:16.749 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi171.stderr:ceph-fuse[31022]: fuse finished with error 0 and tester_r 0 2024-09-13T17:44:16.751 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi171.front.sepia.ceph.com 2024-09-13T17:44:16.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:16.752 DEBUG:teuthology.orchestra.run.smithi171:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-13T17:44:16.821 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:44:16.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.912+0000 7f3a0979d640 1 Processor -- start 2024-09-13T17:44:16.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.912+0000 7f3a0979d640 1 -- start start 2024-09-13T17:44:16.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.913+0000 7f3a0979d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3a04068920 0x7f3a0409d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:16.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.913+0000 7f3a0979d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3a0409e020 0x7f3a0412fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:16.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.913+0000 7f3a0979d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a0409e470 con 0x7f3a0409dc50 2024-09-13T17:44:16.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.913+0000 7f3a0979d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a04068cf0 con 0x7f3a0409e020 2024-09-13T17:44:16.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.913+0000 7f3a0979d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3a040695a0 con 0x7f3a04068920 2024-09-13T17:44:16.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.913+0000 7f3a02ffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3a04068920 0x7f3a0409d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:16.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.913+0000 7f3a037fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3a0409e020 0x7f3a0412fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:16.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.913+0000 7f3a027fc640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f3a0409dc50 0x7f3a0412f310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57242/0 (socket says 172.21.15.102:57242) 2024-09-13T17:44:16.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.913+0000 7f3a027fc640 1 -- 172.21.15.102:0/3288009778 learned_addr learned my addr 172.21.15.102:0/3288009778 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:44:16.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.913+0000 7f3a02ffd640 1 -- 172.21.15.102:0/3288009778 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3a0409e020 msgr2=0x7f3a0412fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:16.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.913+0000 7f3a02ffd640 1 --2- 172.21.15.102:0/3288009778 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3a0409e020 0x7f3a0412fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:16.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.913+0000 7f3a02ffd640 1 -- 172.21.15.102:0/3288009778 >> v1:172.21.15.102:6789/0 conn(0x7f3a0409dc50 legacy=0x7f3a0412f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:16.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.913+0000 7f3a01ffb640 1 -- 172.21.15.102:0/3288009778 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4083767175 0 0) 0x7f3a0409e470 con 0x7f3a0409dc50 2024-09-13T17:44:16.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.913+0000 7f3a02ffd640 1 -- 172.21.15.102:0/3288009778 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a0409f090 con 0x7f3a04068920 2024-09-13T17:44:16.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.913+0000 7f3a037fe640 1 --2- 172.21.15.102:0/3288009778 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3a0409e020 0x7f3a0412fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:16.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.914+0000 7f3a02ffd640 1 --2- 172.21.15.102:0/3288009778 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3a04068920 0x7f3a0409d710 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f39fc009880 tx=0x7f39fc0090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a6cb680e4d1731b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:16.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.914+0000 7f3a01ffb640 1 -- 172.21.15.102:0/3288009778 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc02ac50 con 0x7f3a04068920 2024-09-13T17:44:16.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.914+0000 7f3a01ffb640 1 -- 172.21.15.102:0/3288009778 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39fc02adb0 con 0x7f3a04068920 2024-09-13T17:44:16.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.914+0000 7f3a01ffb640 1 -- 172.21.15.102:0/3288009778 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc032670 con 0x7f3a04068920 2024-09-13T17:44:16.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.915+0000 7f3a0979d640 1 -- 172.21.15.102:0/3288009778 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3a04068920 msgr2=0x7f3a0409d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:16.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.915+0000 7f3a0979d640 1 --2- 172.21.15.102:0/3288009778 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3a04068920 0x7f3a0409d710 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f39fc009880 tx=0x7f39fc0090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:16.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.915+0000 7f3a0979d640 1 -- 172.21.15.102:0/3288009778 shutdown_connections 2024-09-13T17:44:16.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.915+0000 7f3a0979d640 1 --2- 172.21.15.102:0/3288009778 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3a0409e020 0x7f3a0412fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:16.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.915+0000 7f3a0979d640 1 --2- 172.21.15.102:0/3288009778 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3a04068920 0x7f3a0409d710 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:16.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.915+0000 7f3a0979d640 1 -- 172.21.15.102:0/3288009778 >> 172.21.15.102:0/3288009778 conn(0x7f3a0406f320 msgr2=0x7f3a040a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:16.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.915+0000 7f3a0979d640 1 -- 172.21.15.102:0/3288009778 shutdown_connections 2024-09-13T17:44:16.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.915+0000 7f3a0979d640 1 -- 172.21.15.102:0/3288009778 wait complete. 2024-09-13T17:44:16.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.916+0000 7f3a0979d640 1 Processor -- start 2024-09-13T17:44:16.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.916+0000 7f3a0979d640 1 -- start start 2024-09-13T17:44:16.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.916+0000 7f3a0979d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3a04068920 0x7f3a0413eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:16.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.916+0000 7f3a0979d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3a0409dc50 0x7f3a0413f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:16.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.916+0000 7f3a0979d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a0409e020 0x7f3a0413f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:16.923 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.916+0000 7f3a0979d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0411db00 con 0x7f3a0409e020 2024-09-13T17:44:16.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.916+0000 7f3a0979d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0409e620 con 0x7f3a0409dc50 2024-09-13T17:44:16.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.916+0000 7f3a0979d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0411dd30 con 0x7f3a04068920 2024-09-13T17:44:16.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.916+0000 7f3a02ffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3a04068920 0x7f3a0413eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:16.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.916+0000 7f3a02ffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3a04068920 0x7f3a0413eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:57910/0 (socket says 172.21.15.102:57910) 2024-09-13T17:44:16.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.916+0000 7f3a02ffd640 1 -- 172.21.15.102:0/1425259955 learned_addr learned my addr 172.21.15.102:0/1425259955 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:16.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.916+0000 7f3a037fe640 1 --2- 172.21.15.102:0/1425259955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a0409e020 0x7f3a0413f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:16.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.916+0000 7f3a027fc640 1 --2- 172.21.15.102:0/1425259955 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3a0409dc50 0x7f3a0413f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:16.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.917+0000 7f3a02ffd640 1 -- 172.21.15.102:0/1425259955 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3a0409dc50 msgr2=0x7f3a0413f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:16.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.917+0000 7f3a02ffd640 1 --2- 172.21.15.102:0/1425259955 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3a0409dc50 0x7f3a0413f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:16.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.917+0000 7f3a02ffd640 1 -- 172.21.15.102:0/1425259955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a0409e020 msgr2=0x7f3a0413f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:16.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.917+0000 7f3a02ffd640 1 --2- 172.21.15.102:0/1425259955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a0409e020 0x7f3a0413f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:16.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.917+0000 7f3a02ffd640 1 -- 172.21.15.102:0/1425259955 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f4009a20 con 0x7f3a04068920 2024-09-13T17:44:16.924 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.917+0000 7f3a02ffd640 1 --2- 172.21.15.102:0/1425259955 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3a04068920 0x7f3a0413eb00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f39fc02a580 tx=0x7f39fc032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:16.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.917+0000 7f39faffd640 1 -- 172.21.15.102:0/1425259955 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc03b710 con 0x7f3a04068920 2024-09-13T17:44:16.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.917+0000 7f39faffd640 1 -- 172.21.15.102:0/1425259955 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39fc03b870 con 0x7f3a04068920 2024-09-13T17:44:16.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.917+0000 7f3a0979d640 1 -- 172.21.15.102:0/1425259955 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39fc009390 con 0x7f3a04068920 2024-09-13T17:44:16.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.917+0000 7f39faffd640 1 -- 172.21.15.102:0/1425259955 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc03bc30 con 0x7f3a04068920 2024-09-13T17:44:16.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.917+0000 7f3a0979d640 1 -- 172.21.15.102:0/1425259955 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a0411c3c0 con 0x7f3a04068920 2024-09-13T17:44:16.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.918+0000 7f39faffd640 1 -- 172.21.15.102:0/1425259955 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f39fc039040 con 0x7f3a04068920 2024-09-13T17:44:16.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.918+0000 7f39faffd640 1 -- 172.21.15.102:0/1425259955 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3660+0+0 (secure 0 0 0) 0x7f39fc061b80 con 0x7f3a04068920 2024-09-13T17:44:16.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.918+0000 7f3a0979d640 1 -- 172.21.15.102:0/1425259955 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a04064000 con 0x7f3a04068920 2024-09-13T17:44:16.926 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:16.922+0000 7f39faffd640 1 -- 172.21.15.102:0/1425259955 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f39fc042080 con 0x7f3a04068920 2024-09-13T17:44:17.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.110+0000 7f3a0979d640 1 -- 172.21.15.102:0/1425259955 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a0409c7d0 con 0x7f3a04068920 2024-09-13T17:44:17.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.110+0000 7f39faffd640 1 -- 172.21.15.102:0/1425259955 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2268 (secure 0 0 0) 0x7f39fc037030 con 0x7f3a04068920 2024-09-13T17:44:17.112 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:44:17.113 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:44:08.385597+0000","modified":"2024-09-13T17:44:11.073569+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5078},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5078":{"gid":5078,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6817/2209249660","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":2209249660}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-09-13T17:44:17.113 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 11 2024-09-13T17:44:17.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.112+0000 7f3a0979d640 1 -- 172.21.15.102:0/1425259955 >> v1:172.21.15.102:6816/34937 conn(0x7f39e4031160 legacy=0x7f39e4033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:17.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.113+0000 7f3a0979d640 1 -- 172.21.15.102:0/1425259955 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3a04068920 msgr2=0x7f3a0413eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:17.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.113+0000 7f3a0979d640 1 --2- 172.21.15.102:0/1425259955 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3a04068920 0x7f3a0413eb00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f39fc02a580 tx=0x7f39fc032000 comp rx=0 tx=0).stop 2024-09-13T17:44:17.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.113+0000 7f3a0979d640 1 -- 172.21.15.102:0/1425259955 shutdown_connections 2024-09-13T17:44:17.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.113+0000 7f3a0979d640 1 --2- 172.21.15.102:0/1425259955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a0409e020 0x7f3a0413f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:17.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.113+0000 7f3a0979d640 1 --2- 172.21.15.102:0/1425259955 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f3a0409dc50 0x7f3a0413f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:17.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.113+0000 7f3a0979d640 1 --2- 172.21.15.102:0/1425259955 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f3a04068920 0x7f3a0413eb00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:17.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.113+0000 7f3a0979d640 1 -- 172.21.15.102:0/1425259955 >> 172.21.15.102:0/1425259955 conn(0x7f3a0406f320 msgr2=0x7f3a040a0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:17.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.113+0000 7f3a0979d640 1 -- 172.21.15.102:0/1425259955 shutdown_connections 2024-09-13T17:44:17.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.113+0000 7f3a0979d640 1 -- 172.21.15.102:0/1425259955 wait complete. 2024-09-13T17:44:17.125 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:44:17.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.215+0000 7f973e35d640 1 Processor -- start 2024-09-13T17:44:17.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.215+0000 7f973e35d640 1 -- start start 2024-09-13T17:44:17.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.216+0000 7f973e35d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9738068920 0x7f973809d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:17.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.216+0000 7f973e35d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f973809e020 0x7f973812fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:17.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.216+0000 7f973e35d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973809e470 con 0x7f973809dc50 2024-09-13T17:44:17.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.216+0000 7f973e35d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9738068cf0 con 0x7f9738068920 2024-09-13T17:44:17.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.216+0000 7f973e35d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f97380695a0 con 0x7f973809e020 2024-09-13T17:44:17.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.216+0000 7f9737fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9738068920 0x7f973809d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:17.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.216+0000 7f973c8fa640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f973809e020 0x7f973812fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:17.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.216+0000 7f9737fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9738068920 0x7f973809d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35352/0 (socket says 172.21.15.102:35352) 2024-09-13T17:44:17.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.216+0000 7f9737fff640 1 -- 172.21.15.102:0/1931009484 learned_addr learned my addr 172.21.15.102:0/1931009484 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:17.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.216+0000 7f9736ffd640 1 -- 172.21.15.102:0/1931009484 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1465123142 0 0) 0x7f973809e470 con 0x7f973809dc50 2024-09-13T17:44:17.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.216+0000 7f9736ffd640 1 -- 172.21.15.102:0/1931009484 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97240034e0 con 0x7f973809dc50 2024-09-13T17:44:17.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.216+0000 7f9737fff640 1 -- 172.21.15.102:0/1931009484 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f973809e020 msgr2=0x7f973812fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:17.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.216+0000 7f9737fff640 1 --2- 172.21.15.102:0/1931009484 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f973809e020 0x7f973812fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:17.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.216+0000 7f9737fff640 1 -- 172.21.15.102:0/1931009484 >> v1:172.21.15.102:6789/0 conn(0x7f973809dc50 legacy=0x7f973812f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:17.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.216+0000 7f9737fff640 1 -- 172.21.15.102:0/1931009484 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973809f090 con 0x7f9738068920 2024-09-13T17:44:17.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.216+0000 7f973c8fa640 1 --2- 172.21.15.102:0/1931009484 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f973809e020 0x7f973812fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:17.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.217+0000 7f9737fff640 1 --2- 172.21.15.102:0/1931009484 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9738068920 0x7f973809d710 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9728009940 tx=0x7f97280091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dc073e014eec729 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:17.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.217+0000 7f9736ffd640 1 -- 172.21.15.102:0/1931009484 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972802ad00 con 0x7f9738068920 2024-09-13T17:44:17.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.217+0000 7f9736ffd640 1 -- 172.21.15.102:0/1931009484 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f972802ae60 con 0x7f9738068920 2024-09-13T17:44:17.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.217+0000 7f9736ffd640 1 -- 172.21.15.102:0/1931009484 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9728032670 con 0x7f9738068920 2024-09-13T17:44:17.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.218+0000 7f973e35d640 1 -- 172.21.15.102:0/1931009484 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9738068920 msgr2=0x7f973809d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:17.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.218+0000 7f973e35d640 1 --2- 172.21.15.102:0/1931009484 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9738068920 0x7f973809d710 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9728009940 tx=0x7f97280091d0 comp rx=0 tx=0).stop 2024-09-13T17:44:17.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.218+0000 7f973e35d640 1 -- 172.21.15.102:0/1931009484 shutdown_connections 2024-09-13T17:44:17.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.218+0000 7f973e35d640 1 --2- 172.21.15.102:0/1931009484 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f973809e020 0x7f973812fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:17.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.218+0000 7f973e35d640 1 --2- 172.21.15.102:0/1931009484 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9738068920 0x7f973809d710 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:17.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.218+0000 7f973e35d640 1 -- 172.21.15.102:0/1931009484 >> 172.21.15.102:0/1931009484 conn(0x7f973806f320 msgr2=0x7f97380a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:17.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.218+0000 7f973e35d640 1 -- 172.21.15.102:0/1931009484 shutdown_connections 2024-09-13T17:44:17.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.218+0000 7f973e35d640 1 -- 172.21.15.102:0/1931009484 wait complete. 2024-09-13T17:44:17.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.219+0000 7f973e35d640 1 Processor -- start 2024-09-13T17:44:17.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.219+0000 7f973e35d640 1 -- start start 2024-09-13T17:44:17.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.219+0000 7f973e35d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9738068920 0x7f973811e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:17.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.219+0000 7f973e35d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f973809dc50 0x7f973811af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:17.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.219+0000 7f973e35d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f973809e020 0x7f973811b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:17.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.219+0000 7f973e35d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973809e620 con 0x7f973809e020 2024-09-13T17:44:17.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.219+0000 7f973e35d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97380a21a0 con 0x7f973809dc50 2024-09-13T17:44:17.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.219+0000 7f973e35d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f973811dbd0 con 0x7f9738068920 2024-09-13T17:44:17.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.219+0000 7f9737fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9738068920 0x7f973811e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:17.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.219+0000 7f9737fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9738068920 0x7f973811e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:57928/0 (socket says 172.21.15.102:57928) 2024-09-13T17:44:17.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.219+0000 7f9737fff640 1 -- 172.21.15.102:0/16806894 learned_addr learned my addr 172.21.15.102:0/16806894 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:17.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.219+0000 7f97377fe640 1 --2- 172.21.15.102:0/16806894 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f973809dc50 0x7f973811af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:17.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.219+0000 7f973c8fa640 1 --2- 172.21.15.102:0/16806894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f973809e020 0x7f973811b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:17.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.220+0000 7f9737fff640 1 -- 172.21.15.102:0/16806894 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f973809dc50 msgr2=0x7f973811af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:17.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.220+0000 7f9737fff640 1 --2- 172.21.15.102:0/16806894 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f973809dc50 0x7f973811af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:17.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.220+0000 7f9737fff640 1 -- 172.21.15.102:0/16806894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f973809e020 msgr2=0x7f973811b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:17.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.220+0000 7f9737fff640 1 --2- 172.21.15.102:0/16806894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f973809e020 0x7f973811b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:17.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.220+0000 7f9737fff640 1 -- 172.21.15.102:0/16806894 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9730009a20 con 0x7f9738068920 2024-09-13T17:44:17.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.220+0000 7f9737fff640 1 --2- 172.21.15.102:0/16806894 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9738068920 0x7f973811e670 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f97280099a0 tx=0x7f9728032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:17.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.220+0000 7f972ffff640 1 -- 172.21.15.102:0/16806894 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972803b610 con 0x7f9738068920 2024-09-13T17:44:17.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.220+0000 7f972ffff640 1 -- 172.21.15.102:0/16806894 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f972803b770 con 0x7f9738068920 2024-09-13T17:44:17.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.220+0000 7f973e35d640 1 -- 172.21.15.102:0/16806894 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9728009450 con 0x7f9738068920 2024-09-13T17:44:17.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.220+0000 7f972ffff640 1 -- 172.21.15.102:0/16806894 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972803bb30 con 0x7f9738068920 2024-09-13T17:44:17.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.220+0000 7f973e35d640 1 -- 172.21.15.102:0/16806894 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f973811bb80 con 0x7f9738068920 2024-09-13T17:44:17.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.221+0000 7f972ffff640 1 -- 172.21.15.102:0/16806894 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f9728039040 con 0x7f9738068920 2024-09-13T17:44:17.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.221+0000 7f972ffff640 1 -- 172.21.15.102:0/16806894 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3660+0+0 (secure 0 0 0) 0x7f9728041030 con 0x7f9738068920 2024-09-13T17:44:17.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.221+0000 7f973e35d640 1 -- 172.21.15.102:0/16806894 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9738064000 con 0x7f9738068920 2024-09-13T17:44:17.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.225+0000 7f972ffff640 1 -- 172.21.15.102:0/16806894 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f9728037080 con 0x7f9738068920 2024-09-13T17:44:17.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.378+0000 7f973e35d640 1 -- 172.21.15.102:0/16806894 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f973809c7d0 con 0x7f9738068920 2024-09-13T17:44:17.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.379+0000 7f972ffff640 1 -- 172.21.15.102:0/16806894 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+11330 (secure 0 0 0) 0x7f972804b020 con 0x7f9738068920 2024-09-13T17:44:17.381 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:44:17.381 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":32,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:44:10.068600+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:44:06.718328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-13T17:44:07.355624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:44:17.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.381+0000 7f973e35d640 1 -- 172.21.15.102:0/16806894 >> v1:172.21.15.102:6816/34937 conn(0x7f9718031160 legacy=0x7f9718033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:17.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.381+0000 7f973e35d640 1 -- 172.21.15.102:0/16806894 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9738068920 msgr2=0x7f973811e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:17.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.381+0000 7f973e35d640 1 --2- 172.21.15.102:0/16806894 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9738068920 0x7f973811e670 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f97280099a0 tx=0x7f9728032000 comp rx=0 tx=0).stop 2024-09-13T17:44:17.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.381+0000 7f973e35d640 1 -- 172.21.15.102:0/16806894 shutdown_connections 2024-09-13T17:44:17.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.381+0000 7f973e35d640 1 --2- 172.21.15.102:0/16806894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f973809e020 0x7f973811b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:17.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.381+0000 7f973e35d640 1 --2- 172.21.15.102:0/16806894 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f973809dc50 0x7f973811af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:17.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.381+0000 7f973e35d640 1 --2- 172.21.15.102:0/16806894 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9738068920 0x7f973811e670 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:17.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.381+0000 7f973e35d640 1 -- 172.21.15.102:0/16806894 >> 172.21.15.102:0/16806894 conn(0x7f973806f320 msgr2=0x7f9738132620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:17.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.381+0000 7f973e35d640 1 -- 172.21.15.102:0/16806894 shutdown_connections 2024-09-13T17:44:17.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.381+0000 7f973e35d640 1 -- 172.21.15.102:0/16806894 wait complete. 2024-09-13T17:44:17.394 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:54.442612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T17:41:59.076782+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 16, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:44:06.718328+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:44:07.355624+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:44:17.395 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:44:17.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.488+0000 7fe79391d640 1 Processor -- start 2024-09-13T17:44:17.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.489+0000 7fe79391d640 1 -- start start 2024-09-13T17:44:17.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.489+0000 7fe79391d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe78c097530 0x7fe78c097900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:17.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.489+0000 7fe79391d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe78c098210 0x7fe78c133640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:17.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.489+0000 7fe79391d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe78c098c50 con 0x7fe78c097e40 2024-09-13T17:44:17.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.489+0000 7fe79391d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe78c06ed00 con 0x7fe78c098210 2024-09-13T17:44:17.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.489+0000 7fe79391d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c068bd0 con 0x7fe78c097530 2024-09-13T17:44:17.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.489+0000 7fe7916b9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe78c097530 0x7fe78c097900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:17.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.489+0000 7fe791eba640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fe78c098210 0x7fe78c133640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:17.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.489+0000 7fe7916b9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe78c097530 0x7fe78c097900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:57940/0 (socket says 172.21.15.102:57940) 2024-09-13T17:44:17.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.489+0000 7fe7916b9640 1 -- 172.21.15.102:0/373542924 learned_addr learned my addr 172.21.15.102:0/373542924 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:17.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.489+0000 7fe790eb8640 1 --1- 172.21.15.102:0/373542924 >> v1:172.21.15.102:6789/0 conn(0x7fe78c097e40 0x7fe78c12ff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57270/0 (socket says 172.21.15.102:57270) 2024-09-13T17:44:17.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.490+0000 7fe791eba640 1 -- 172.21.15.102:0/373542924 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe78c097530 msgr2=0x7fe78c097900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:17.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.490+0000 7fe791eba640 1 --2- 172.21.15.102:0/373542924 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe78c097530 0x7fe78c097900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:17.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.490+0000 7fe791eba640 1 -- 172.21.15.102:0/373542924 >> v1:172.21.15.102:6789/0 conn(0x7fe78c097e40 legacy=0x7fe78c12ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:17.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.490+0000 7fe78bfff640 1 -- 172.21.15.102:0/373542924 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3306015512 0 0) 0x7fe78c098c50 con 0x7fe78c097e40 2024-09-13T17:44:17.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.490+0000 7fe791eba640 1 -- 172.21.15.102:0/373542924 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe78c099040 con 0x7fe78c098210 2024-09-13T17:44:17.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.490+0000 7fe7916b9640 1 --2- 172.21.15.102:0/373542924 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe78c097530 0x7fe78c097900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:17.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.490+0000 7fe791eba640 1 --2- 172.21.15.102:0/373542924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe78c098210 0x7fe78c133640 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe77c009e60 tx=0x7fe77c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=408db47b2c52655e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:17.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.490+0000 7fe78bfff640 1 -- 172.21.15.102:0/373542924 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77c007550 con 0x7fe78c098210 2024-09-13T17:44:17.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.490+0000 7fe78bfff640 1 -- 172.21.15.102:0/373542924 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe77c02d050 con 0x7fe78c098210 2024-09-13T17:44:17.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.490+0000 7fe78bfff640 1 -- 172.21.15.102:0/373542924 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77c0078e0 con 0x7fe78c098210 2024-09-13T17:44:17.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.491+0000 7fe79391d640 1 -- 172.21.15.102:0/373542924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe78c098210 msgr2=0x7fe78c133640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:17.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.491+0000 7fe79391d640 1 --2- 172.21.15.102:0/373542924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe78c098210 0x7fe78c133640 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe77c009e60 tx=0x7fe77c009780 comp rx=0 tx=0).stop 2024-09-13T17:44:17.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.491+0000 7fe79391d640 1 -- 172.21.15.102:0/373542924 shutdown_connections 2024-09-13T17:44:17.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.491+0000 7fe79391d640 1 --2- 172.21.15.102:0/373542924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe78c098210 0x7fe78c133640 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:17.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.491+0000 7fe79391d640 1 --2- 172.21.15.102:0/373542924 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe78c097530 0x7fe78c097900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:17.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.491+0000 7fe79391d640 1 -- 172.21.15.102:0/373542924 >> 172.21.15.102:0/373542924 conn(0x7fe78c09e720 msgr2=0x7fe78c09eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:17.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.491+0000 7fe79391d640 1 -- 172.21.15.102:0/373542924 shutdown_connections 2024-09-13T17:44:17.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.491+0000 7fe79391d640 1 -- 172.21.15.102:0/373542924 wait complete. 2024-09-13T17:44:17.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.492+0000 7fe79391d640 1 Processor -- start 2024-09-13T17:44:17.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.492+0000 7fe79391d640 1 -- start start 2024-09-13T17:44:17.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.492+0000 7fe79391d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe78c097530 0x7fe78c12e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:17.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.492+0000 7fe79391d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe78c097e40 0x7fe78c11a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:17.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.492+0000 7fe79391d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe78c098210 0x7fe78c11ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:17.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.492+0000 7fe79391d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c11c6f0 con 0x7fe78c097e40 2024-09-13T17:44:17.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.492+0000 7fe79391d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c11d130 con 0x7fe78c097530 2024-09-13T17:44:17.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.492+0000 7fe79391d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c11c1b0 con 0x7fe78c098210 2024-09-13T17:44:17.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.493+0000 7fe7916b9640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe78c097530 0x7fe78c12e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:17.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.493+0000 7fe791eba640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe78c098210 0x7fe78c11ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:17.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.493+0000 7fe790eb8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe78c097e40 0x7fe78c11a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:17.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.493+0000 7fe7916b9640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe78c097530 0x7fe78c12e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35364/0 (socket says 172.21.15.102:35364) 2024-09-13T17:44:17.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.493+0000 7fe7916b9640 1 -- 172.21.15.102:0/3478549036 learned_addr learned my addr 172.21.15.102:0/3478549036 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:17.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.493+0000 7fe791eba640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe78c098210 0x7fe78c11ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:57942/0 (socket says 172.21.15.102:57942) 2024-09-13T17:44:17.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.493+0000 7fe7916b9640 1 -- 172.21.15.102:0/3478549036 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe78c098210 msgr2=0x7fe78c11ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:17.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.493+0000 7fe7916b9640 1 --2- 172.21.15.102:0/3478549036 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe78c098210 0x7fe78c11ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:17.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.493+0000 7fe7916b9640 1 -- 172.21.15.102:0/3478549036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe78c097e40 msgr2=0x7fe78c11a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:17.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.493+0000 7fe7916b9640 1 --2- 172.21.15.102:0/3478549036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe78c097e40 0x7fe78c11a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:17.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.493+0000 7fe7916b9640 1 -- 172.21.15.102:0/3478549036 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe78400b6f0 con 0x7fe78c097530 2024-09-13T17:44:17.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.493+0000 7fe791eba640 1 --2- 172.21.15.102:0/3478549036 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe78c098210 0x7fe78c11ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:17.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.493+0000 7fe790eb8640 1 --2- 172.21.15.102:0/3478549036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe78c097e40 0x7fe78c11a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:17.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.493+0000 7fe7916b9640 1 --2- 172.21.15.102:0/3478549036 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe78c097530 0x7fe78c12e620 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe784012f00 tx=0x7fe784008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:17.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.493+0000 7fe7897fa640 1 -- 172.21.15.102:0/3478549036 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7840174a0 con 0x7fe78c097530 2024-09-13T17:44:17.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.493+0000 7fe7897fa640 1 -- 172.21.15.102:0/3478549036 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe784017600 con 0x7fe78c097530 2024-09-13T17:44:17.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.493+0000 7fe7897fa640 1 -- 172.21.15.102:0/3478549036 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe784017980 con 0x7fe78c097530 2024-09-13T17:44:17.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.493+0000 7fe79391d640 1 -- 172.21.15.102:0/3478549036 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe77c009a20 con 0x7fe78c097530 2024-09-13T17:44:17.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.493+0000 7fe79391d640 1 -- 172.21.15.102:0/3478549036 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe78c11e810 con 0x7fe78c097530 2024-09-13T17:44:17.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.494+0000 7fe7897fa640 1 -- 172.21.15.102:0/3478549036 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fe784017d30 con 0x7fe78c097530 2024-09-13T17:44:17.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.494+0000 7fe7897fa640 1 -- 172.21.15.102:0/3478549036 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3660+0+0 (secure 0 0 0) 0x7fe784046620 con 0x7fe78c097530 2024-09-13T17:44:17.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.494+0000 7fe79391d640 1 -- 172.21.15.102:0/3478549036 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe78c11def0 con 0x7fe78c097530 2024-09-13T17:44:17.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.499+0000 7fe7897fa640 1 -- 172.21.15.102:0/3478549036 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fe784021070 con 0x7fe78c097530 2024-09-13T17:44:17.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.648+0000 7fe79391d640 1 -- 172.21.15.102:0/3478549036 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe78c064000 con 0x7fe78c097530 2024-09-13T17:44:17.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.649+0000 7fe7897fa640 1 -- 172.21.15.102:0/3478549036 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v32) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe78402b0a0 con 0x7fe78c097530 2024-09-13T17:44:17.651 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-09-13T17:44:17.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.651+0000 7fe79391d640 1 -- 172.21.15.102:0/3478549036 >> v1:172.21.15.102:6816/34937 conn(0x7fe778031160 legacy=0x7fe778033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:17.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.651+0000 7fe79391d640 1 -- 172.21.15.102:0/3478549036 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe78c097530 msgr2=0x7fe78c12e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:17.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.651+0000 7fe79391d640 1 --2- 172.21.15.102:0/3478549036 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe78c097530 0x7fe78c12e620 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe784012f00 tx=0x7fe784008040 comp rx=0 tx=0).stop 2024-09-13T17:44:17.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.651+0000 7fe79391d640 1 -- 172.21.15.102:0/3478549036 shutdown_connections 2024-09-13T17:44:17.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.651+0000 7fe79391d640 1 --2- 172.21.15.102:0/3478549036 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fe78c098210 0x7fe78c11ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:17.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.651+0000 7fe79391d640 1 --2- 172.21.15.102:0/3478549036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe78c097e40 0x7fe78c11a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:17.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.651+0000 7fe79391d640 1 --2- 172.21.15.102:0/3478549036 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fe78c097530 0x7fe78c12e620 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:17.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.651+0000 7fe79391d640 1 -- 172.21.15.102:0/3478549036 >> 172.21.15.102:0/3478549036 conn(0x7fe78c09e720 msgr2=0x7fe78c09c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:17.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.651+0000 7fe79391d640 1 -- 172.21.15.102:0/3478549036 shutdown_connections 2024-09-13T17:44:17.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.651+0000 7fe79391d640 1 -- 172.21.15.102:0/3478549036 wait complete. 2024-09-13T17:44:17.663 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-13T17:44:17.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.757+0000 7fd992b1d640 1 Processor -- start 2024-09-13T17:44:17.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.757+0000 7fd992b1d640 1 -- start start 2024-09-13T17:44:17.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.757+0000 7fd992b1d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd98c096a70 0x7fd98c0987c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:17.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.757+0000 7fd992b1d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd98c098d00 0x7fd98c12c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:17.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.757+0000 7fd992b1d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd98c099110 0x7fd98c12c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:17.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.757+0000 7fd992b1d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd98c069c30 con 0x7fd98c096a70 2024-09-13T17:44:17.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.757+0000 7fd992b1d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd98c06ed00 con 0x7fd98c099110 2024-09-13T17:44:17.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.757+0000 7fd992b1d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd98c0999e0 con 0x7fd98c098d00 2024-09-13T17:44:17.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.757+0000 7fd98bfff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd98c098d00 0x7fd98c12c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:17.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.757+0000 7fd9908b9640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd98c096a70 0x7fd98c0987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:17.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.757+0000 7fd9910ba640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd98c099110 0x7fd98c12c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:17.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.757+0000 7fd98bfff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd98c098d00 0x7fd98c12c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:57948/0 (socket says 172.21.15.102:57948) 2024-09-13T17:44:17.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.757+0000 7fd98bfff640 1 -- 172.21.15.102:0/1847638947 learned_addr learned my addr 172.21.15.102:0/1847638947 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:17.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.758+0000 7fd98bfff640 1 -- 172.21.15.102:0/1847638947 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd98c099110 msgr2=0x7fd98c12c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:17.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.758+0000 7fd98bfff640 1 --2- 172.21.15.102:0/1847638947 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd98c099110 0x7fd98c12c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:17.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.758+0000 7fd98bfff640 1 -- 172.21.15.102:0/1847638947 >> v2:172.21.15.102:3300/0 conn(0x7fd98c096a70 msgr2=0x7fd98c0987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:17.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.758+0000 7fd98bfff640 1 --2- 172.21.15.102:0/1847638947 >> v2:172.21.15.102:3300/0 conn(0x7fd98c096a70 0x7fd98c0987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:17.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.758+0000 7fd98bfff640 1 -- 172.21.15.102:0/1847638947 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd98c0997a0 con 0x7fd98c098d00 2024-09-13T17:44:17.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.758+0000 7fd9908b9640 1 --2- 172.21.15.102:0/1847638947 >> v2:172.21.15.102:3300/0 conn(0x7fd98c096a70 0x7fd98c0987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:17.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.758+0000 7fd9910ba640 1 --2- 172.21.15.102:0/1847638947 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd98c099110 0x7fd98c12c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:17.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.758+0000 7fd98bfff640 1 --2- 172.21.15.102:0/1847638947 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd98c098d00 0x7fd98c12c2e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd97c00f4c0 tx=0x7fd97c00dc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45bfe0f259942a18 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:17.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.758+0000 7fd98b7fe640 1 -- 172.21.15.102:0/1847638947 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c00d2e0 con 0x7fd98c098d00 2024-09-13T17:44:17.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.758+0000 7fd98b7fe640 1 -- 172.21.15.102:0/1847638947 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd97c00d440 con 0x7fd98c098d00 2024-09-13T17:44:17.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.758+0000 7fd98b7fe640 1 -- 172.21.15.102:0/1847638947 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c00b810 con 0x7fd98c098d00 2024-09-13T17:44:17.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.759+0000 7fd992b1d640 1 -- 172.21.15.102:0/1847638947 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd98c098d00 msgr2=0x7fd98c12c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:17.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.759+0000 7fd992b1d640 1 --2- 172.21.15.102:0/1847638947 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd98c098d00 0x7fd98c12c2e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd97c00f4c0 tx=0x7fd97c00dc50 comp rx=0 tx=0).stop 2024-09-13T17:44:17.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.759+0000 7fd992b1d640 1 -- 172.21.15.102:0/1847638947 shutdown_connections 2024-09-13T17:44:17.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.759+0000 7fd992b1d640 1 --2- 172.21.15.102:0/1847638947 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd98c099110 0x7fd98c12c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:17.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.759+0000 7fd992b1d640 1 --2- 172.21.15.102:0/1847638947 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd98c098d00 0x7fd98c12c2e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:17.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.759+0000 7fd992b1d640 1 --2- 172.21.15.102:0/1847638947 >> v2:172.21.15.102:3300/0 conn(0x7fd98c096a70 0x7fd98c0987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:17.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.759+0000 7fd992b1d640 1 -- 172.21.15.102:0/1847638947 >> 172.21.15.102:0/1847638947 conn(0x7fd98c09e720 msgr2=0x7fd98c09eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:17.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.759+0000 7fd992b1d640 1 -- 172.21.15.102:0/1847638947 shutdown_connections 2024-09-13T17:44:17.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.759+0000 7fd992b1d640 1 -- 172.21.15.102:0/1847638947 wait complete. 2024-09-13T17:44:17.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.760+0000 7fd992b1d640 1 Processor -- start 2024-09-13T17:44:17.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.760+0000 7fd992b1d640 1 -- start start 2024-09-13T17:44:17.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.760+0000 7fd992b1d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd98c096a70 0x7fd98c11b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:17.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.760+0000 7fd992b1d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd98c098d00 0x7fd98c117a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:17.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.760+0000 7fd992b1d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd98c099110 0x7fd98c117fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:17.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.760+0000 7fd992b1d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98c1197c0 con 0x7fd98c096a70 2024-09-13T17:44:17.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.760+0000 7fd992b1d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd98c12a750 con 0x7fd98c099110 2024-09-13T17:44:17.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.760+0000 7fd992b1d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd98c11a620 con 0x7fd98c098d00 2024-09-13T17:44:17.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.760+0000 7fd9908b9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd98c096a70 0x7fd98c11b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:17.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.760+0000 7fd98bfff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd98c098d00 0x7fd98c117a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:17.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.760+0000 7fd9908b9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd98c096a70 0x7fd98c11b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57144/0 (socket says 172.21.15.102:57144) 2024-09-13T17:44:17.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.761+0000 7fd9910ba640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd98c099110 0x7fd98c117fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:17.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.761+0000 7fd9908b9640 1 -- 172.21.15.102:0/403466481 learned_addr learned my addr 172.21.15.102:0/403466481 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:17.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.761+0000 7fd9910ba640 1 -- 172.21.15.102:0/403466481 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd98c098d00 msgr2=0x7fd98c117a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:17.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.761+0000 7fd9910ba640 1 --2- 172.21.15.102:0/403466481 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd98c098d00 0x7fd98c117a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:17.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.761+0000 7fd9910ba640 1 -- 172.21.15.102:0/403466481 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd98c096a70 msgr2=0x7fd98c11b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:17.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.761+0000 7fd9910ba640 1 --2- 172.21.15.102:0/403466481 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd98c096a70 0x7fd98c11b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:17.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.761+0000 7fd9910ba640 1 -- 172.21.15.102:0/403466481 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd98400b440 con 0x7fd98c099110 2024-09-13T17:44:17.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.761+0000 7fd9908b9640 1 --2- 172.21.15.102:0/403466481 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd98c096a70 0x7fd98c11b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:17.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.761+0000 7fd98bfff640 1 --2- 172.21.15.102:0/403466481 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd98c098d00 0x7fd98c117a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:17.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.761+0000 7fd9910ba640 1 --2- 172.21.15.102:0/403466481 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd98c099110 0x7fd98c117fd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd97400e980 tx=0x7fd9740090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:17.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.761+0000 7fd988ff9640 1 -- 172.21.15.102:0/403466481 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97400c940 con 0x7fd98c099110 2024-09-13T17:44:17.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.761+0000 7fd988ff9640 1 -- 172.21.15.102:0/403466481 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd97400caa0 con 0x7fd98c099110 2024-09-13T17:44:17.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.761+0000 7fd988ff9640 1 -- 172.21.15.102:0/403466481 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97400ce60 con 0x7fd98c099110 2024-09-13T17:44:17.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.761+0000 7fd992b1d640 1 -- 172.21.15.102:0/403466481 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd974009390 con 0x7fd98c099110 2024-09-13T17:44:17.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.761+0000 7fd992b1d640 1 -- 172.21.15.102:0/403466481 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd97c00f050 con 0x7fd98c099110 2024-09-13T17:44:17.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.762+0000 7fd988ff9640 1 -- 172.21.15.102:0/403466481 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fd974013680 con 0x7fd98c099110 2024-09-13T17:44:17.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.762+0000 7fd988ff9640 1 -- 172.21.15.102:0/403466481 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3660+0+0 (secure 0 0 0) 0x7fd9740420e0 con 0x7fd98c099110 2024-09-13T17:44:17.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.762+0000 7fd992b1d640 1 -- 172.21.15.102:0/403466481 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd98c064000 con 0x7fd98c099110 2024-09-13T17:44:17.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.766+0000 7fd988ff9640 1 -- 172.21.15.102:0/403466481 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fd974020080 con 0x7fd98c099110 2024-09-13T17:44:17.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.916+0000 7fd992b1d640 1 -- 172.21.15.102:0/403466481 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd98c11a760 con 0x7fd98c099110 2024-09-13T17:44:17.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.916+0000 7fd988ff9640 1 -- 172.21.15.102:0/403466481 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v32) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd97401ab40 con 0x7fd98c099110 2024-09-13T17:44:17.919 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 8 2024-09-13T17:44:17.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.918+0000 7fd992b1d640 1 -- 172.21.15.102:0/403466481 >> v1:172.21.15.102:6816/34937 conn(0x7fd96c031190 legacy=0x7fd96c033640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:17.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.918+0000 7fd992b1d640 1 -- 172.21.15.102:0/403466481 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd98c099110 msgr2=0x7fd98c117fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:17.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.918+0000 7fd992b1d640 1 --2- 172.21.15.102:0/403466481 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd98c099110 0x7fd98c117fd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd97400e980 tx=0x7fd9740090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:17.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.918+0000 7fd992b1d640 1 -- 172.21.15.102:0/403466481 shutdown_connections 2024-09-13T17:44:17.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.918+0000 7fd992b1d640 1 --2- 172.21.15.102:0/403466481 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd98c099110 0x7fd98c117fd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:17.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.918+0000 7fd992b1d640 1 --2- 172.21.15.102:0/403466481 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd98c098d00 0x7fd98c117a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:17.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.918+0000 7fd992b1d640 1 --2- 172.21.15.102:0/403466481 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd98c096a70 0x7fd98c11b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:17.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.918+0000 7fd992b1d640 1 -- 172.21.15.102:0/403466481 >> 172.21.15.102:0/403466481 conn(0x7fd98c09e720 msgr2=0x7fd98c0971f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:17.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.918+0000 7fd992b1d640 1 -- 172.21.15.102:0/403466481 shutdown_connections 2024-09-13T17:44:17.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:17.918+0000 7fd992b1d640 1 -- 172.21.15.102:0/403466481 wait complete. 2024-09-13T17:44:17.931 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:44:18.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.026+0000 7f4706834640 1 Processor -- start 2024-09-13T17:44:18.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.026+0000 7f4706834640 1 -- start start 2024-09-13T17:44:18.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.027+0000 7f4706834640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f470009ad20 0x7f470012fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:18.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.027+0000 7f4706834640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f470012ff60 0x7f4700096a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:18.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.027+0000 7f4706834640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f470006df00 0x7f4700096fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:18.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.027+0000 7f4706834640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f470006e9e0 con 0x7f470012ff60 2024-09-13T17:44:18.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.027+0000 7f4706834640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4700098980 con 0x7f470006df00 2024-09-13T17:44:18.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.027+0000 7f4706834640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4700097f00 con 0x7f470009ad20 2024-09-13T17:44:18.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.027+0000 7f46f7fff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f470012ff60 0x7f4700096a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:18.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.027+0000 7f4704dd1640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f470006df00 0x7f4700096fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:18.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.027+0000 7f46fffff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f470009ad20 0x7f470012fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:18.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.027+0000 7f46f7fff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f470012ff60 0x7f4700096a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57158/0 (socket says 172.21.15.102:57158) 2024-09-13T17:44:18.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.027+0000 7f46f7fff640 1 -- 172.21.15.102:0/4083234760 learned_addr learned my addr 172.21.15.102:0/4083234760 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:18.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.027+0000 7f46f7fff640 1 -- 172.21.15.102:0/4083234760 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f470009ad20 msgr2=0x7f470012fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:18.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.027+0000 7f46f7fff640 1 --2- 172.21.15.102:0/4083234760 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f470009ad20 0x7f470012fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.027+0000 7f46f7fff640 1 -- 172.21.15.102:0/4083234760 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f470006df00 msgr2=0x7f4700096fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:18.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.027+0000 7f46f7fff640 1 --2- 172.21.15.102:0/4083234760 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f470006df00 0x7f4700096fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.027+0000 7f46f7fff640 1 -- 172.21.15.102:0/4083234760 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f470012f710 con 0x7f470012ff60 2024-09-13T17:44:18.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.027+0000 7f46fffff640 1 --2- 172.21.15.102:0/4083234760 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f470009ad20 0x7f470012fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:18.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.028+0000 7f46f7fff640 1 --2- 172.21.15.102:0/4083234760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f470012ff60 0x7f4700096a70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f46e8009910 tx=0x7f46e80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d57512499c365130 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:18.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.028+0000 7f46ff7fe640 1 -- 172.21.15.102:0/4083234760 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e802ad20 con 0x7f470012ff60 2024-09-13T17:44:18.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.028+0000 7f46ff7fe640 1 -- 172.21.15.102:0/4083234760 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e802ae80 con 0x7f470012ff60 2024-09-13T17:44:18.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.028+0000 7f46ff7fe640 1 -- 172.21.15.102:0/4083234760 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e8032670 con 0x7f470012ff60 2024-09-13T17:44:18.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.029+0000 7f4706834640 1 -- 172.21.15.102:0/4083234760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f470012ff60 msgr2=0x7f4700096a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:18.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.029+0000 7f4706834640 1 --2- 172.21.15.102:0/4083234760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f470012ff60 0x7f4700096a70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f46e8009910 tx=0x7f46e80091d0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.029+0000 7f4706834640 1 -- 172.21.15.102:0/4083234760 shutdown_connections 2024-09-13T17:44:18.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.029+0000 7f4706834640 1 --2- 172.21.15.102:0/4083234760 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f470006df00 0x7f4700096fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.029+0000 7f4706834640 1 --2- 172.21.15.102:0/4083234760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f470012ff60 0x7f4700096a70 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.029+0000 7f4706834640 1 --2- 172.21.15.102:0/4083234760 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f470009ad20 0x7f470012fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.029+0000 7f4706834640 1 -- 172.21.15.102:0/4083234760 >> 172.21.15.102:0/4083234760 conn(0x7f470009e6b0 msgr2=0x7f470009eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:18.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.029+0000 7f4706834640 1 -- 172.21.15.102:0/4083234760 shutdown_connections 2024-09-13T17:44:18.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.029+0000 7f4706834640 1 -- 172.21.15.102:0/4083234760 wait complete. 2024-09-13T17:44:18.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.029+0000 7f4706834640 1 Processor -- start 2024-09-13T17:44:18.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.030+0000 7f4706834640 1 -- start start 2024-09-13T17:44:18.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.030+0000 7f4706834640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f470006df00 0x7f470011d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:18.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.030+0000 7f4706834640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f470009ad20 0x7f4700119af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:18.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.030+0000 7f4706834640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f470012ff60 0x7f470011a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:18.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.030+0000 7f4706834640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470009cbf0 con 0x7f470009ad20 2024-09-13T17:44:18.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.030+0000 7f4706834640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f470011c570 con 0x7f470006df00 2024-09-13T17:44:18.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.030+0000 7f4706834640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f470011c820 con 0x7f470012ff60 2024-09-13T17:44:18.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.030+0000 7f46fffff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f470006df00 0x7f470011d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:18.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.030+0000 7f46f7fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f470009ad20 0x7f4700119af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:18.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.030+0000 7f46f7fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f470009ad20 0x7f4700119af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57166/0 (socket says 172.21.15.102:57166) 2024-09-13T17:44:18.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.030+0000 7f4704dd1640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f470012ff60 0x7f470011a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:18.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.030+0000 7f46fffff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f470006df00 0x7f470011d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35386/0 (socket says 172.21.15.102:35386) 2024-09-13T17:44:18.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.030+0000 7f46f7fff640 1 -- 172.21.15.102:0/3563690843 learned_addr learned my addr 172.21.15.102:0/3563690843 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:18.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.030+0000 7f46fffff640 1 -- 172.21.15.102:0/3563690843 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f470012ff60 msgr2=0x7f470011a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:18.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.030+0000 7f46fffff640 1 --2- 172.21.15.102:0/3563690843 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f470012ff60 0x7f470011a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.030+0000 7f46fffff640 1 -- 172.21.15.102:0/3563690843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f470009ad20 msgr2=0x7f4700119af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:18.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.030+0000 7f46fffff640 1 --2- 172.21.15.102:0/3563690843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f470009ad20 0x7f4700119af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.037 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.030+0000 7f46fffff640 1 -- 172.21.15.102:0/3563690843 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f0009390 con 0x7f470006df00 2024-09-13T17:44:18.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.030+0000 7f46fffff640 1 --2- 172.21.15.102:0/3563690843 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f470006df00 0x7f470011d2c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f46f000ea50 tx=0x7f46f00090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:18.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.031+0000 7f46fcff9640 1 -- 172.21.15.102:0/3563690843 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f000c990 con 0x7f470006df00 2024-09-13T17:44:18.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.031+0000 7f46fcff9640 1 -- 172.21.15.102:0/3563690843 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f000caf0 con 0x7f470006df00 2024-09-13T17:44:18.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.031+0000 7f4706834640 1 -- 172.21.15.102:0/3563690843 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e8009450 con 0x7f470006df00 2024-09-13T17:44:18.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.031+0000 7f46fcff9640 1 -- 172.21.15.102:0/3563690843 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f000ce90 con 0x7f470006df00 2024-09-13T17:44:18.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.031+0000 7f4706834640 1 -- 172.21.15.102:0/3563690843 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f470011a570 con 0x7f470006df00 2024-09-13T17:44:18.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.031+0000 7f46fcff9640 1 -- 172.21.15.102:0/3563690843 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f46f0013680 con 0x7f470006df00 2024-09-13T17:44:18.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.032+0000 7f46fcff9640 1 -- 172.21.15.102:0/3563690843 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3660+0+0 (secure 0 0 0) 0x7f46f0042330 con 0x7f470006df00 2024-09-13T17:44:18.038 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.032+0000 7f4706834640 1 -- 172.21.15.102:0/3563690843 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4700064000 con 0x7f470006df00 2024-09-13T17:44:18.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.036+0000 7f46fcff9640 1 -- 172.21.15.102:0/3563690843 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f46f0021020 con 0x7f470006df00 2024-09-13T17:44:18.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.190+0000 7f4706834640 1 -- 172.21.15.102:0/3563690843 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f470011a7c0 con 0x7f470006df00 2024-09-13T17:44:18.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.190+0000 7f46fcff9640 1 -- 172.21.15.102:0/3563690843 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v32) v1 ==== 104+0+11 (secure 0 0 0) 0x7f46f0023ad0 con 0x7f470006df00 2024-09-13T17:44:18.192 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-09-13T17:44:18.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.192+0000 7f4706834640 1 -- 172.21.15.102:0/3563690843 >> v1:172.21.15.102:6816/34937 conn(0x7f46e0031160 legacy=0x7f46e0033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:18.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.192+0000 7f4706834640 1 -- 172.21.15.102:0/3563690843 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f470006df00 msgr2=0x7f470011d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:18.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.192+0000 7f4706834640 1 --2- 172.21.15.102:0/3563690843 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f470006df00 0x7f470011d2c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f46f000ea50 tx=0x7f46f00090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.192+0000 7f4706834640 1 -- 172.21.15.102:0/3563690843 shutdown_connections 2024-09-13T17:44:18.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.192+0000 7f4706834640 1 --2- 172.21.15.102:0/3563690843 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f470012ff60 0x7f470011a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.192+0000 7f4706834640 1 --2- 172.21.15.102:0/3563690843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f470009ad20 0x7f4700119af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.192+0000 7f4706834640 1 --2- 172.21.15.102:0/3563690843 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f470006df00 0x7f470011d2c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.192+0000 7f4706834640 1 -- 172.21.15.102:0/3563690843 >> 172.21.15.102:0/3563690843 conn(0x7f470009e6b0 msgr2=0x7f4700099ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:18.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.192+0000 7f4706834640 1 -- 172.21.15.102:0/3563690843 shutdown_connections 2024-09-13T17:44:18.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.192+0000 7f4706834640 1 -- 172.21.15.102:0/3563690843 wait complete. 2024-09-13T17:44:18.204 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:44:18.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.298+0000 7f6b36694640 1 Processor -- start 2024-09-13T17:44:18.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.298+0000 7f6b36694640 1 -- start start 2024-09-13T17:44:18.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.299+0000 7f6b36694640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6b30098010 0x7f6b30130f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:18.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.299+0000 7f6b36694640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6b300983e0 0x7f6b30133470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:18.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.299+0000 7f6b36694640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b30098c50 con 0x7f6b30097530 2024-09-13T17:44:18.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.299+0000 7f6b36694640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b3006ed00 con 0x7f6b300983e0 2024-09-13T17:44:18.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.299+0000 7f6b36694640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b30068bd0 con 0x7f6b30098010 2024-09-13T17:44:18.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.299+0000 7f6b27fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6b30098010 0x7f6b30130f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:18.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.299+0000 7f6b34c31640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6b300983e0 0x7f6b30133470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:18.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.299+0000 7f6b34c31640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6b300983e0 0x7f6b30133470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57180/0 (socket says 172.21.15.102:57180) 2024-09-13T17:44:18.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.299+0000 7f6b27fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6b30098010 0x7f6b30130f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35392/0 (socket says 172.21.15.102:35392) 2024-09-13T17:44:18.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.299+0000 7f6b34c31640 1 -- 172.21.15.102:0/3946355770 learned_addr learned my addr 172.21.15.102:0/3946355770 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:18.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.299+0000 7f6b2ffff640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f6b30097530 0x7f6b30097900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57286/0 (socket says 172.21.15.102:57286) 2024-09-13T17:44:18.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.299+0000 7f6b34c31640 1 -- 172.21.15.102:0/3946355770 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6b30098010 msgr2=0x7f6b30130f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:18.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.299+0000 7f6b34c31640 1 --2- 172.21.15.102:0/3946355770 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6b30098010 0x7f6b30130f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.299+0000 7f6b34c31640 1 -- 172.21.15.102:0/3946355770 >> v1:172.21.15.102:6789/0 conn(0x7f6b30097530 legacy=0x7f6b30097900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:18.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.299+0000 7f6b34c31640 1 -- 172.21.15.102:0/3946355770 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b30099040 con 0x7f6b300983e0 2024-09-13T17:44:18.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.299+0000 7f6b2f7fe640 1 -- 172.21.15.102:0/3946355770 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3674267427 0 0) 0x7f6b30098c50 con 0x7f6b30097530 2024-09-13T17:44:18.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.299+0000 7f6b27fff640 1 --2- 172.21.15.102:0/3946355770 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6b30098010 0x7f6b30130f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:18.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.300+0000 7f6b34c31640 1 --2- 172.21.15.102:0/3946355770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b300983e0 0x7f6b30133470 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800bfc0 tx=0x7f6b2800d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17415b40aa96fd4c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:18.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.300+0000 7f6b2f7fe640 1 -- 172.21.15.102:0/3946355770 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2800b2e0 con 0x7f6b300983e0 2024-09-13T17:44:18.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.300+0000 7f6b2f7fe640 1 -- 172.21.15.102:0/3946355770 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2802d050 con 0x7f6b300983e0 2024-09-13T17:44:18.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.300+0000 7f6b2f7fe640 1 -- 172.21.15.102:0/3946355770 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b28009840 con 0x7f6b300983e0 2024-09-13T17:44:18.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.301+0000 7f6b36694640 1 -- 172.21.15.102:0/3946355770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b300983e0 msgr2=0x7f6b30133470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:18.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.301+0000 7f6b36694640 1 --2- 172.21.15.102:0/3946355770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b300983e0 0x7f6b30133470 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800bfc0 tx=0x7f6b2800d040 comp rx=0 tx=0).stop 2024-09-13T17:44:18.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.301+0000 7f6b36694640 1 -- 172.21.15.102:0/3946355770 shutdown_connections 2024-09-13T17:44:18.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.301+0000 7f6b36694640 1 --2- 172.21.15.102:0/3946355770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b300983e0 0x7f6b30133470 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.301+0000 7f6b36694640 1 --2- 172.21.15.102:0/3946355770 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6b30098010 0x7f6b30130f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.301+0000 7f6b36694640 1 -- 172.21.15.102:0/3946355770 >> 172.21.15.102:0/3946355770 conn(0x7f6b3009e720 msgr2=0x7f6b3009eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:18.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.301+0000 7f6b36694640 1 -- 172.21.15.102:0/3946355770 shutdown_connections 2024-09-13T17:44:18.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.301+0000 7f6b36694640 1 -- 172.21.15.102:0/3946355770 wait complete. 2024-09-13T17:44:18.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.302+0000 7f6b36694640 1 Processor -- start 2024-09-13T17:44:18.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.302+0000 7f6b36694640 1 -- start start 2024-09-13T17:44:18.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.302+0000 7f6b36694640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b30097530 0x7f6b3012b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:18.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.302+0000 7f6b36694640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6b30098010 0x7f6b30117620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:18.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.302+0000 7f6b36694640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6b300983e0 0x7f6b30117b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:18.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.302+0000 7f6b36694640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b30119690 con 0x7f6b30097530 2024-09-13T17:44:18.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.302+0000 7f6b36694640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b301192d0 con 0x7f6b30098010 2024-09-13T17:44:18.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.302+0000 7f6b36694640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3011a060 con 0x7f6b300983e0 2024-09-13T17:44:18.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.302+0000 7f6b2ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b30097530 0x7f6b3012b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:18.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.302+0000 7f6b27fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6b30098010 0x7f6b30117620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:18.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.302+0000 7f6b27fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6b30098010 0x7f6b30117620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35396/0 (socket says 172.21.15.102:35396) 2024-09-13T17:44:18.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.302+0000 7f6b27fff640 1 -- 172.21.15.102:0/90700833 learned_addr learned my addr 172.21.15.102:0/90700833 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:18.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.302+0000 7f6b34c31640 1 --2- 172.21.15.102:0/90700833 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6b300983e0 0x7f6b30117b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:18.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.302+0000 7f6b2ffff640 1 -- 172.21.15.102:0/90700833 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6b300983e0 msgr2=0x7f6b30117b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:18.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.302+0000 7f6b2ffff640 1 --2- 172.21.15.102:0/90700833 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6b300983e0 0x7f6b30117b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.302+0000 7f6b2ffff640 1 -- 172.21.15.102:0/90700833 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6b30098010 msgr2=0x7f6b30117620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:18.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.302+0000 7f6b2ffff640 1 --2- 172.21.15.102:0/90700833 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6b30098010 0x7f6b30117620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.302+0000 7f6b2ffff640 1 -- 172.21.15.102:0/90700833 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1800d830 con 0x7f6b30097530 2024-09-13T17:44:18.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.303+0000 7f6b2ffff640 1 --2- 172.21.15.102:0/90700833 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b30097530 0x7f6b3012b5c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6b20000c00 tx=0x7f6b20007ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:18.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.303+0000 7f6b2cff9640 1 -- 172.21.15.102:0/90700833 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b200072e0 con 0x7f6b30097530 2024-09-13T17:44:18.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.303+0000 7f6b2cff9640 1 -- 172.21.15.102:0/90700833 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b20007460 con 0x7f6b30097530 2024-09-13T17:44:18.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.303+0000 7f6b36694640 1 -- 172.21.15.102:0/90700833 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b2800bb00 con 0x7f6b30097530 2024-09-13T17:44:18.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.303+0000 7f6b2cff9640 1 -- 172.21.15.102:0/90700833 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b200049f0 con 0x7f6b30097530 2024-09-13T17:44:18.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.303+0000 7f6b36694640 1 -- 172.21.15.102:0/90700833 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b301198d0 con 0x7f6b30097530 2024-09-13T17:44:18.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.304+0000 7f6b2cff9640 1 -- 172.21.15.102:0/90700833 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f6b20013400 con 0x7f6b30097530 2024-09-13T17:44:18.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.304+0000 7f6b2cff9640 1 -- 172.21.15.102:0/90700833 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3660+0+0 (secure 0 0 0) 0x7f6b20041470 con 0x7f6b30097530 2024-09-13T17:44:18.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.304+0000 7f6b36694640 1 -- 172.21.15.102:0/90700833 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b3006a230 con 0x7f6b30097530 2024-09-13T17:44:18.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.308+0000 7f6b2cff9640 1 -- 172.21.15.102:0/90700833 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f6b3006a230 con 0x7f6b30097530 2024-09-13T17:44:18.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.461+0000 7f6b36694640 1 -- 172.21.15.102:0/90700833 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6b3006a230 con 0x7f6b30097530 2024-09-13T17:44:18.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.462+0000 7f6b2cff9640 1 -- 172.21.15.102:0/90700833 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v32) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6b3006a230 con 0x7f6b30097530 2024-09-13T17:44:18.464 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 64 2024-09-13T17:44:18.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.464+0000 7f6b36694640 1 -- 172.21.15.102:0/90700833 >> v1:172.21.15.102:6816/34937 conn(0x7f6b10031160 legacy=0x7f6b10033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:18.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.464+0000 7f6b36694640 1 -- 172.21.15.102:0/90700833 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b30097530 msgr2=0x7f6b3012b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:18.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.464+0000 7f6b36694640 1 --2- 172.21.15.102:0/90700833 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b30097530 0x7f6b3012b5c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6b20000c00 tx=0x7f6b20007ca0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.464+0000 7f6b36694640 1 -- 172.21.15.102:0/90700833 shutdown_connections 2024-09-13T17:44:18.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.464+0000 7f6b36694640 1 --2- 172.21.15.102:0/90700833 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6b300983e0 0x7f6b30117b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.464+0000 7f6b36694640 1 --2- 172.21.15.102:0/90700833 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f6b30098010 0x7f6b30117620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.464+0000 7f6b36694640 1 --2- 172.21.15.102:0/90700833 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b30097530 0x7f6b3012b5c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.464+0000 7f6b36694640 1 -- 172.21.15.102:0/90700833 >> 172.21.15.102:0/90700833 conn(0x7f6b3009e720 msgr2=0x7f6b3012e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:18.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.464+0000 7f6b36694640 1 -- 172.21.15.102:0/90700833 shutdown_connections 2024-09-13T17:44:18.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.464+0000 7f6b36694640 1 -- 172.21.15.102:0/90700833 wait complete. 2024-09-13T17:44:18.477 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:44:18.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.568+0000 7f442388b640 1 Processor -- start 2024-09-13T17:44:18.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.568+0000 7f442388b640 1 -- start start 2024-09-13T17:44:18.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.569+0000 7f442388b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f441c067a80 0x7f441c12e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:18.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.569+0000 7f442388b640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f441c12ef80 0x7f441c12da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:18.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.569+0000 7f442388b640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f441c09cb10 con 0x7f441c12eb70 2024-09-13T17:44:18.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.569+0000 7f442388b640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f441c067e50 con 0x7f441c12ef80 2024-09-13T17:44:18.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.569+0000 7f442388b640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f441c068ed0 con 0x7f441c067a80 2024-09-13T17:44:18.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.569+0000 7f4421e28640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f441c12ef80 0x7f441c12da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:18.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.569+0000 7f4420e26640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f441c12eb70 0x7f441c12d380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57300/0 (socket says 172.21.15.102:57300) 2024-09-13T17:44:18.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.569+0000 7f4420e26640 1 -- 172.21.15.102:0/1681311435 learned_addr learned my addr 172.21.15.102:0/1681311435 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:44:18.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.569+0000 7f4421627640 1 --2- 172.21.15.102:0/1681311435 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f441c067a80 0x7f441c12e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:18.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.569+0000 7f4421e28640 1 -- 172.21.15.102:0/1681311435 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f441c067a80 msgr2=0x7f441c12e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:18.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.569+0000 7f4421e28640 1 --2- 172.21.15.102:0/1681311435 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f441c067a80 0x7f441c12e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.569+0000 7f4421e28640 1 -- 172.21.15.102:0/1681311435 >> v1:172.21.15.102:6789/0 conn(0x7f441c12eb70 legacy=0x7f441c12d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:18.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.569+0000 7f4421e28640 1 -- 172.21.15.102:0/1681311435 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f441c06a050 con 0x7f441c12ef80 2024-09-13T17:44:18.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.569+0000 7f4413fff640 1 -- 172.21.15.102:0/1681311435 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2717564835 0 0) 0x7f441c09cb10 con 0x7f441c12eb70 2024-09-13T17:44:18.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.569+0000 7f4421627640 1 --2- 172.21.15.102:0/1681311435 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f441c067a80 0x7f441c12e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:18.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.570+0000 7f4421e28640 1 --2- 172.21.15.102:0/1681311435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f441c12ef80 0x7f441c12da90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4414009850 tx=0x7f44140090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=408fc856bd157553 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:18.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.570+0000 7f4413fff640 1 -- 172.21.15.102:0/1681311435 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441402ac80 con 0x7f441c12ef80 2024-09-13T17:44:18.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.570+0000 7f4413fff640 1 -- 172.21.15.102:0/1681311435 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f441402ade0 con 0x7f441c12ef80 2024-09-13T17:44:18.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.570+0000 7f4413fff640 1 -- 172.21.15.102:0/1681311435 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4414032670 con 0x7f441c12ef80 2024-09-13T17:44:18.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.570+0000 7f442388b640 1 -- 172.21.15.102:0/1681311435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f441c12ef80 msgr2=0x7f441c12da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:18.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.570+0000 7f442388b640 1 --2- 172.21.15.102:0/1681311435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f441c12ef80 0x7f441c12da90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4414009850 tx=0x7f44140090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.571+0000 7f442388b640 1 -- 172.21.15.102:0/1681311435 shutdown_connections 2024-09-13T17:44:18.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.571+0000 7f442388b640 1 --2- 172.21.15.102:0/1681311435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f441c12ef80 0x7f441c12da90 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.571+0000 7f442388b640 1 --2- 172.21.15.102:0/1681311435 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f441c067a80 0x7f441c12e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.571+0000 7f442388b640 1 -- 172.21.15.102:0/1681311435 >> 172.21.15.102:0/1681311435 conn(0x7f441c0a1300 msgr2=0x7f441c067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:18.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.571+0000 7f442388b640 1 -- 172.21.15.102:0/1681311435 shutdown_connections 2024-09-13T17:44:18.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.571+0000 7f442388b640 1 -- 172.21.15.102:0/1681311435 wait complete. 2024-09-13T17:44:18.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.571+0000 7f442388b640 1 Processor -- start 2024-09-13T17:44:18.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.571+0000 7f442388b640 1 -- start start 2024-09-13T17:44:18.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.572+0000 7f442388b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f441c067a80 0x7f441c11bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:18.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.572+0000 7f442388b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f441c12eb70 0x7f441c11c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:18.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.572+0000 7f442388b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f441c12ef80 0x7f441c13e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:18.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.572+0000 7f442388b640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c0688b0 con 0x7f441c067a80 2024-09-13T17:44:18.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.572+0000 7f442388b640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f441c11ea80 con 0x7f441c12ef80 2024-09-13T17:44:18.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.572+0000 7f442388b640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f441c11ed40 con 0x7f441c12eb70 2024-09-13T17:44:18.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.572+0000 7f4421627640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f441c067a80 0x7f441c11bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:18.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.572+0000 7f4420e26640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f441c12eb70 0x7f441c11c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:18.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.572+0000 7f4421627640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f441c067a80 0x7f441c11bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57220/0 (socket says 172.21.15.102:57220) 2024-09-13T17:44:18.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.572+0000 7f4421627640 1 -- 172.21.15.102:0/2714701545 learned_addr learned my addr 172.21.15.102:0/2714701545 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:18.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.572+0000 7f4421e28640 1 --2- 172.21.15.102:0/2714701545 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f441c12ef80 0x7f441c13e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:18.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.572+0000 7f4420e26640 1 -- 172.21.15.102:0/2714701545 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f441c12ef80 msgr2=0x7f441c13e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:18.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.572+0000 7f4420e26640 1 --2- 172.21.15.102:0/2714701545 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f441c12ef80 0x7f441c13e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.572+0000 7f4420e26640 1 -- 172.21.15.102:0/2714701545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f441c067a80 msgr2=0x7f441c11bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:18.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.572+0000 7f4420e26640 1 --2- 172.21.15.102:0/2714701545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f441c067a80 0x7f441c11bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.572+0000 7f4420e26640 1 -- 172.21.15.102:0/2714701545 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4418009a20 con 0x7f441c12eb70 2024-09-13T17:44:18.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.572+0000 7f4421627640 1 --2- 172.21.15.102:0/2714701545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f441c067a80 0x7f441c11bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:18.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.573+0000 7f4420e26640 1 --2- 172.21.15.102:0/2714701545 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f441c12eb70 0x7f441c11c2a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f441c068150 tx=0x7f440c007c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:18.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.573+0000 7f44117fa640 1 -- 172.21.15.102:0/2714701545 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c005aa0 con 0x7f441c12eb70 2024-09-13T17:44:18.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.573+0000 7f44117fa640 1 -- 172.21.15.102:0/2714701545 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f440c005c40 con 0x7f441c12eb70 2024-09-13T17:44:18.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.573+0000 7f442388b640 1 -- 172.21.15.102:0/2714701545 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4414009390 con 0x7f441c12eb70 2024-09-13T17:44:18.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.573+0000 7f44117fa640 1 -- 172.21.15.102:0/2714701545 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c0070d0 con 0x7f441c12eb70 2024-09-13T17:44:18.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.573+0000 7f442388b640 1 -- 172.21.15.102:0/2714701545 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f441c11f770 con 0x7f441c12eb70 2024-09-13T17:44:18.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.574+0000 7f44117fa640 1 -- 172.21.15.102:0/2714701545 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f440c007500 con 0x7f441c12eb70 2024-09-13T17:44:18.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.574+0000 7f44117fa640 1 -- 172.21.15.102:0/2714701545 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3660+0+0 (secure 0 0 0) 0x7f440c041400 con 0x7f441c12eb70 2024-09-13T17:44:18.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.574+0000 7f442388b640 1 -- 172.21.15.102:0/2714701545 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f441c064000 con 0x7f441c12eb70 2024-09-13T17:44:18.586 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.578+0000 7f44117fa640 1 -- 172.21.15.102:0/2714701545 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f440c017020 con 0x7f441c12eb70 2024-09-13T17:44:18.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.763+0000 7f442388b640 1 -- 172.21.15.102:0/2714701545 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f441c09e860 con 0x7f441c12eb70 2024-09-13T17:44:18.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.764+0000 7f44117fa640 1 -- 172.21.15.102:0/2714701545 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2268 (secure 0 0 0) 0x7f440c00e4c0 con 0x7f441c12eb70 2024-09-13T17:44:18.766 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:44:18.766 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:44:08.385597+0000","modified":"2024-09-13T17:44:11.073569+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5078},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5078":{"gid":5078,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6817/2209249660","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":2209249660}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-09-13T17:44:18.766 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 11 2024-09-13T17:44:18.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.766+0000 7f442388b640 1 -- 172.21.15.102:0/2714701545 >> v1:172.21.15.102:6816/34937 conn(0x7f4400031160 legacy=0x7f4400033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:18.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.766+0000 7f442388b640 1 -- 172.21.15.102:0/2714701545 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f441c12eb70 msgr2=0x7f441c11c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:18.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.766+0000 7f442388b640 1 --2- 172.21.15.102:0/2714701545 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f441c12eb70 0x7f441c11c2a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f441c068150 tx=0x7f440c007c80 comp rx=0 tx=0).stop 2024-09-13T17:44:18.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.766+0000 7f442388b640 1 -- 172.21.15.102:0/2714701545 shutdown_connections 2024-09-13T17:44:18.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.766+0000 7f442388b640 1 --2- 172.21.15.102:0/2714701545 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f441c12ef80 0x7f441c13e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.766+0000 7f442388b640 1 --2- 172.21.15.102:0/2714701545 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f441c12eb70 0x7f441c11c2a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.766+0000 7f442388b640 1 --2- 172.21.15.102:0/2714701545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f441c067a80 0x7f441c11bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.766+0000 7f442388b640 1 -- 172.21.15.102:0/2714701545 >> 172.21.15.102:0/2714701545 conn(0x7f441c0a1300 msgr2=0x7f441c06dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:18.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.766+0000 7f442388b640 1 -- 172.21.15.102:0/2714701545 shutdown_connections 2024-09-13T17:44:18.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.766+0000 7f442388b640 1 -- 172.21.15.102:0/2714701545 wait complete. 2024-09-13T17:44:18.778 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:44:18.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.869+0000 7f7ed4c34640 1 Processor -- start 2024-09-13T17:44:18.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.869+0000 7f7ed4c34640 1 -- start start 2024-09-13T17:44:18.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.869+0000 7f7ed4c34640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7ed006b140 0x7f7ed00998c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:18.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.869+0000 7f7ed4c34640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f7ed009ac20 0x7f7ed0099e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:18.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.869+0000 7f7ed4c34640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ed006ee90 con 0x7f7ed009a340 2024-09-13T17:44:18.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.869+0000 7f7ed4c34640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ed006ac90 con 0x7f7ed009ac20 2024-09-13T17:44:18.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.869+0000 7f7ed4c34640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed0099510 con 0x7f7ed006b140 2024-09-13T17:44:18.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.869+0000 7f7ecdd9b640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f7ed009ac20 0x7f7ed0099e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:18.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.869+0000 7f7eced9d640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f7ed009a340 0x7f7ed012d280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57310/0 (socket says 172.21.15.102:57310) 2024-09-13T17:44:18.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.869+0000 7f7ece59c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7ed006b140 0x7f7ed00998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:18.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.869+0000 7f7eced9d640 1 -- 172.21.15.102:0/1145973138 learned_addr learned my addr 172.21.15.102:0/1145973138 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:44:18.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.870+0000 7f7ecdd9b640 1 -- 172.21.15.102:0/1145973138 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7ed006b140 msgr2=0x7f7ed00998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:18.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.870+0000 7f7ecdd9b640 1 --2- 172.21.15.102:0/1145973138 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7ed006b140 0x7f7ed00998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.870+0000 7f7ecdd9b640 1 -- 172.21.15.102:0/1145973138 >> v1:172.21.15.102:6789/0 conn(0x7f7ed009a340 legacy=0x7f7ed012d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:18.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.870+0000 7f7ecdd9b640 1 -- 172.21.15.102:0/1145973138 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed0099680 con 0x7f7ed009ac20 2024-09-13T17:44:18.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.870+0000 7f7ecd59a640 1 -- 172.21.15.102:0/1145973138 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 257825847 0 0) 0x7f7ed006ee90 con 0x7f7ed009a340 2024-09-13T17:44:18.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.870+0000 7f7ece59c640 1 --2- 172.21.15.102:0/1145973138 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7ed006b140 0x7f7ed00998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:18.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.870+0000 7f7ecdd9b640 1 --2- 172.21.15.102:0/1145973138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ed009ac20 0x7f7ed0099e00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4009850 tx=0x7f7ec40090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d56af1e36ab4801 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:18.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.871+0000 7f7ecd59a640 1 -- 172.21.15.102:0/1145973138 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec402ac80 con 0x7f7ed009ac20 2024-09-13T17:44:18.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.871+0000 7f7ecd59a640 1 -- 172.21.15.102:0/1145973138 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec402ade0 con 0x7f7ed009ac20 2024-09-13T17:44:18.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.871+0000 7f7ecd59a640 1 -- 172.21.15.102:0/1145973138 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4032670 con 0x7f7ed009ac20 2024-09-13T17:44:18.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.871+0000 7f7ed4c34640 1 -- 172.21.15.102:0/1145973138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ed009ac20 msgr2=0x7f7ed0099e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:18.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.871+0000 7f7ed4c34640 1 --2- 172.21.15.102:0/1145973138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ed009ac20 0x7f7ed0099e00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4009850 tx=0x7f7ec40090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.871+0000 7f7ed4c34640 1 -- 172.21.15.102:0/1145973138 shutdown_connections 2024-09-13T17:44:18.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.871+0000 7f7ed4c34640 1 --2- 172.21.15.102:0/1145973138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ed009ac20 0x7f7ed0099e00 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.871+0000 7f7ed4c34640 1 --2- 172.21.15.102:0/1145973138 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7ed006b140 0x7f7ed00998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.871+0000 7f7ed4c34640 1 -- 172.21.15.102:0/1145973138 >> 172.21.15.102:0/1145973138 conn(0x7f7ed009e820 msgr2=0x7f7ed009ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:18.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.871+0000 7f7ed4c34640 1 -- 172.21.15.102:0/1145973138 shutdown_connections 2024-09-13T17:44:18.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.871+0000 7f7ed4c34640 1 -- 172.21.15.102:0/1145973138 wait complete. 2024-09-13T17:44:18.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.872+0000 7f7ed4c34640 1 Processor -- start 2024-09-13T17:44:18.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.872+0000 7f7ed4c34640 1 -- start start 2024-09-13T17:44:18.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.872+0000 7f7ed4c34640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ed006b140 0x7f7ed012c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:18.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.872+0000 7f7ed4c34640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7ed009a340 0x7f7ed0096a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:18.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.872+0000 7f7ed4c34640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7ed009ac20 0x7f7ed0096fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:18.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.872+0000 7f7ed4c34640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed0120b10 con 0x7f7ed006b140 2024-09-13T17:44:18.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.872+0000 7f7ed4c34640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed0121320 con 0x7f7ed009a340 2024-09-13T17:44:18.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.872+0000 7f7ed4c34640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed00985d0 con 0x7f7ed009ac20 2024-09-13T17:44:18.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.873+0000 7f7ece59c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ed006b140 0x7f7ed012c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:18.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.873+0000 7f7eced9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7ed009ac20 0x7f7ed0096fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:18.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.873+0000 7f7ecdd9b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7ed009a340 0x7f7ed0096a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:18.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.873+0000 7f7ece59c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ed006b140 0x7f7ed012c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57232/0 (socket says 172.21.15.102:57232) 2024-09-13T17:44:18.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.873+0000 7f7ece59c640 1 -- 172.21.15.102:0/3504073434 learned_addr learned my addr 172.21.15.102:0/3504073434 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:18.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.873+0000 7f7eced9d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7ed009ac20 0x7f7ed0096fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:58024/0 (socket says 172.21.15.102:58024) 2024-09-13T17:44:18.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.873+0000 7f7ece59c640 1 -- 172.21.15.102:0/3504073434 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7ed009ac20 msgr2=0x7f7ed0096fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:18.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.873+0000 7f7ece59c640 1 --2- 172.21.15.102:0/3504073434 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7ed009ac20 0x7f7ed0096fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.873+0000 7f7ece59c640 1 -- 172.21.15.102:0/3504073434 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7ed009a340 msgr2=0x7f7ed0096a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:18.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.873+0000 7f7ece59c640 1 --2- 172.21.15.102:0/3504073434 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7ed009a340 0x7f7ed0096a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.873+0000 7f7ece59c640 1 -- 172.21.15.102:0/3504073434 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec0009390 con 0x7f7ed006b140 2024-09-13T17:44:18.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.873+0000 7f7eced9d640 1 --2- 172.21.15.102:0/3504073434 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7ed009ac20 0x7f7ed0096fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:18.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.873+0000 7f7ecdd9b640 1 --2- 172.21.15.102:0/3504073434 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7ed009a340 0x7f7ed0096a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:18.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.873+0000 7f7ece59c640 1 --2- 172.21.15.102:0/3504073434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ed006b140 0x7f7ed012c7c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7ec000ea70 tx=0x7f7ec00090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:18.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.873+0000 7f7ebe7fc640 1 -- 172.21.15.102:0/3504073434 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec000ca10 con 0x7f7ed006b140 2024-09-13T17:44:18.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.873+0000 7f7ebe7fc640 1 -- 172.21.15.102:0/3504073434 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec000cb70 con 0x7f7ed006b140 2024-09-13T17:44:18.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.873+0000 7f7ed4c34640 1 -- 172.21.15.102:0/3504073434 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec4009390 con 0x7f7ed006b140 2024-09-13T17:44:18.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.873+0000 7f7ebe7fc640 1 -- 172.21.15.102:0/3504073434 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0013470 con 0x7f7ed006b140 2024-09-13T17:44:18.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.873+0000 7f7ed4c34640 1 -- 172.21.15.102:0/3504073434 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ed0121a70 con 0x7f7ed006b140 2024-09-13T17:44:18.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.874+0000 7f7ebe7fc640 1 -- 172.21.15.102:0/3504073434 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f7ec0013820 con 0x7f7ed006b140 2024-09-13T17:44:18.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.874+0000 7f7ed4c34640 1 -- 172.21.15.102:0/3504073434 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ed0064000 con 0x7f7ed006b140 2024-09-13T17:44:18.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.874+0000 7f7ebe7fc640 1 -- 172.21.15.102:0/3504073434 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3660+0+0 (secure 0 0 0) 0x7f7ec0042460 con 0x7f7ed006b140 2024-09-13T17:44:18.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:18.878+0000 7f7ebe7fc640 1 -- 172.21.15.102:0/3504073434 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f7ec00236b0 con 0x7f7ed006b140 2024-09-13T17:44:19.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.031+0000 7f7ed4c34640 1 -- 172.21.15.102:0/3504073434 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ed0120c60 con 0x7f7ed006b140 2024-09-13T17:44:19.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.031+0000 7f7ebe7fc640 1 -- 172.21.15.102:0/3504073434 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+11330 (secure 0 0 0) 0x7f7ec00232f0 con 0x7f7ed006b140 2024-09-13T17:44:19.033 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:44:19.034 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":32,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:44:10.068600+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:44:06.718328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-13T17:44:07.355624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:44:19.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.033+0000 7f7ed4c34640 1 -- 172.21.15.102:0/3504073434 >> v1:172.21.15.102:6816/34937 conn(0x7f7eb0031160 legacy=0x7f7eb0033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:19.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.033+0000 7f7ed4c34640 1 -- 172.21.15.102:0/3504073434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ed006b140 msgr2=0x7f7ed012c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:19.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.033+0000 7f7ed4c34640 1 --2- 172.21.15.102:0/3504073434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ed006b140 0x7f7ed012c7c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7ec000ea70 tx=0x7f7ec00090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.033+0000 7f7ed4c34640 1 -- 172.21.15.102:0/3504073434 shutdown_connections 2024-09-13T17:44:19.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.033+0000 7f7ed4c34640 1 --2- 172.21.15.102:0/3504073434 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7ed009ac20 0x7f7ed0096fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.033+0000 7f7ed4c34640 1 --2- 172.21.15.102:0/3504073434 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7ed009a340 0x7f7ed0096a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.033+0000 7f7ed4c34640 1 --2- 172.21.15.102:0/3504073434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ed006b140 0x7f7ed012c7c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.033+0000 7f7ed4c34640 1 -- 172.21.15.102:0/3504073434 >> 172.21.15.102:0/3504073434 conn(0x7f7ed009e820 msgr2=0x7f7ed006cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:19.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.033+0000 7f7ed4c34640 1 -- 172.21.15.102:0/3504073434 shutdown_connections 2024-09-13T17:44:19.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.033+0000 7f7ed4c34640 1 -- 172.21.15.102:0/3504073434 wait complete. 2024-09-13T17:44:19.047 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T17:44:19.047 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:44:19.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.140+0000 7f9ae408b640 1 Processor -- start 2024-09-13T17:44:19.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.140+0000 7f9ae408b640 1 -- start start 2024-09-13T17:44:19.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.140+0000 7f9ae408b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9adc09abb0 0x7f9adc12c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:19.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.140+0000 7f9ae408b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9adc12c6a0 0x7f9adc12cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:19.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.140+0000 7f9ae408b640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9adc06a530 con 0x7f9adc096d80 2024-09-13T17:44:19.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.140+0000 7f9ae408b640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc09af80 con 0x7f9adc09abb0 2024-09-13T17:44:19.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.140+0000 7f9ae408b640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc12cff0 con 0x7f9adc12c6a0 2024-09-13T17:44:19.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.140+0000 7f9ae1626640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9adc12c6a0 0x7f9adc12cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:19.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.140+0000 7f9ae1e27640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9adc09abb0 0x7f9adc12c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:19.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.140+0000 7f9ae2628640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f9adc096d80 0x7f9adc097220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57316/0 (socket says 172.21.15.102:57316) 2024-09-13T17:44:19.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.140+0000 7f9ae2628640 1 -- 172.21.15.102:0/2349925290 learned_addr learned my addr 172.21.15.102:0/2349925290 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:44:19.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.141+0000 7f9ae0e25640 1 -- 172.21.15.102:0/2349925290 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4007069052 0 0) 0x7f9adc06a530 con 0x7f9adc096d80 2024-09-13T17:44:19.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.141+0000 7f9ae1e27640 1 -- 172.21.15.102:0/2349925290 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9adc12c6a0 msgr2=0x7f9adc12cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:19.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.141+0000 7f9ae1e27640 1 --2- 172.21.15.102:0/2349925290 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9adc12c6a0 0x7f9adc12cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.141+0000 7f9ae1e27640 1 -- 172.21.15.102:0/2349925290 >> v1:172.21.15.102:6789/0 conn(0x7f9adc096d80 legacy=0x7f9adc097220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:19.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.141+0000 7f9ae1e27640 1 -- 172.21.15.102:0/2349925290 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9adc098be0 con 0x7f9adc09abb0 2024-09-13T17:44:19.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.141+0000 7f9ae1626640 1 --2- 172.21.15.102:0/2349925290 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9adc12c6a0 0x7f9adc12cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:19.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.141+0000 7f9ae1e27640 1 --2- 172.21.15.102:0/2349925290 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9adc09abb0 0x7f9adc12c160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9ad400bb50 tx=0x7f9ad400b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22979ae19414d077 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:19.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.141+0000 7f9ae0e25640 1 -- 172.21.15.102:0/2349925290 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4031510 con 0x7f9adc09abb0 2024-09-13T17:44:19.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.142+0000 7f9ae0e25640 1 -- 172.21.15.102:0/2349925290 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad402f050 con 0x7f9adc09abb0 2024-09-13T17:44:19.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.142+0000 7f9ae0e25640 1 -- 172.21.15.102:0/2349925290 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad40318a0 con 0x7f9adc09abb0 2024-09-13T17:44:19.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.142+0000 7f9ae408b640 1 -- 172.21.15.102:0/2349925290 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9adc09abb0 msgr2=0x7f9adc12c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:19.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.142+0000 7f9ae408b640 1 --2- 172.21.15.102:0/2349925290 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9adc09abb0 0x7f9adc12c160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9ad400bb50 tx=0x7f9ad400b3b0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.142+0000 7f9ae408b640 1 -- 172.21.15.102:0/2349925290 shutdown_connections 2024-09-13T17:44:19.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.142+0000 7f9ae408b640 1 --2- 172.21.15.102:0/2349925290 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9adc12c6a0 0x7f9adc12cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.142+0000 7f9ae408b640 1 --2- 172.21.15.102:0/2349925290 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9adc09abb0 0x7f9adc12c160 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.142+0000 7f9ae408b640 1 -- 172.21.15.102:0/2349925290 >> 172.21.15.102:0/2349925290 conn(0x7f9adc09e820 msgr2=0x7f9adc09ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:19.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.142+0000 7f9ae408b640 1 -- 172.21.15.102:0/2349925290 shutdown_connections 2024-09-13T17:44:19.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.142+0000 7f9ae408b640 1 -- 172.21.15.102:0/2349925290 wait complete. 2024-09-13T17:44:19.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.143+0000 7f9ae408b640 1 Processor -- start 2024-09-13T17:44:19.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.143+0000 7f9ae408b640 1 -- start start 2024-09-13T17:44:19.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.143+0000 7f9ae408b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9adc096d80 0x7f9adc13a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:19.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.143+0000 7f9ae408b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9adc09abb0 0x7f9adc13ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:19.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.143+0000 7f9ae408b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9adc12c6a0 0x7f9adc115360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:19.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.143+0000 7f9ae408b640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc0981a0 con 0x7f9adc12c6a0 2024-09-13T17:44:19.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.143+0000 7f9ae408b640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc118dd0 con 0x7f9adc09abb0 2024-09-13T17:44:19.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.143+0000 7f9ae408b640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc09d9f0 con 0x7f9adc096d80 2024-09-13T17:44:19.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.143+0000 7f9ae1e27640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9adc096d80 0x7f9adc13a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:19.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.143+0000 7f9ae1626640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9adc09abb0 0x7f9adc13ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:19.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.143+0000 7f9ae1e27640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9adc096d80 0x7f9adc13a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:58044/0 (socket says 172.21.15.102:58044) 2024-09-13T17:44:19.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.143+0000 7f9ae1e27640 1 -- 172.21.15.102:0/1326910663 learned_addr learned my addr 172.21.15.102:0/1326910663 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:19.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.143+0000 7f9ae2628640 1 --2- 172.21.15.102:0/1326910663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9adc12c6a0 0x7f9adc115360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:19.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.144+0000 7f9ae1e27640 1 -- 172.21.15.102:0/1326910663 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9adc09abb0 msgr2=0x7f9adc13ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:19.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.144+0000 7f9ae1e27640 1 --2- 172.21.15.102:0/1326910663 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9adc09abb0 0x7f9adc13ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.144+0000 7f9ae1e27640 1 -- 172.21.15.102:0/1326910663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9adc12c6a0 msgr2=0x7f9adc115360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:19.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.144+0000 7f9ae1e27640 1 --2- 172.21.15.102:0/1326910663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9adc12c6a0 0x7f9adc115360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.144+0000 7f9ae1e27640 1 -- 172.21.15.102:0/1326910663 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad0009390 con 0x7f9adc096d80 2024-09-13T17:44:19.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.144+0000 7f9ae1626640 1 --2- 172.21.15.102:0/1326910663 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9adc09abb0 0x7f9adc13ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:19.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.144+0000 7f9ae2628640 1 --2- 172.21.15.102:0/1326910663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9adc12c6a0 0x7f9adc115360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:19.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.144+0000 7f9ae1e27640 1 --2- 172.21.15.102:0/1326910663 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9adc096d80 0x7f9adc13a7e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9ad400bb80 tx=0x7f9ad4039000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:19.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.144+0000 7f9ad9ffb640 1 -- 172.21.15.102:0/1326910663 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4031890 con 0x7f9adc096d80 2024-09-13T17:44:19.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.144+0000 7f9ad9ffb640 1 -- 172.21.15.102:0/1326910663 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad4033af0 con 0x7f9adc096d80 2024-09-13T17:44:19.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.144+0000 7f9ad9ffb640 1 -- 172.21.15.102:0/1326910663 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4047460 con 0x7f9adc096d80 2024-09-13T17:44:19.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.144+0000 7f9ae408b640 1 -- 172.21.15.102:0/1326910663 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ad400b630 con 0x7f9adc096d80 2024-09-13T17:44:19.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.144+0000 7f9ae408b640 1 -- 172.21.15.102:0/1326910663 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9adc116f30 con 0x7f9adc096d80 2024-09-13T17:44:19.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.145+0000 7f9ad9ffb640 1 -- 172.21.15.102:0/1326910663 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f9ad4047890 con 0x7f9adc096d80 2024-09-13T17:44:19.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.145+0000 7f9ad9ffb640 1 -- 172.21.15.102:0/1326910663 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3660+0+0 (secure 0 0 0) 0x7f9ad40658c0 con 0x7f9adc096d80 2024-09-13T17:44:19.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.145+0000 7f9ae408b640 1 -- 172.21.15.102:0/1326910663 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9adc098e00 con 0x7f9adc096d80 2024-09-13T17:44:19.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.150+0000 7f9ad9ffb640 1 -- 172.21.15.102:0/1326910663 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f9ad404c070 con 0x7f9adc096d80 2024-09-13T17:44:19.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.331+0000 7f9ae408b640 1 -- 172.21.15.102:0/1326910663 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9adc064000 con 0x7f9adc096d80 2024-09-13T17:44:19.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.331+0000 7f9ad9ffb640 1 -- 172.21.15.102:0/1326910663 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2268 (secure 0 0 0) 0x7f9ad4042030 con 0x7f9adc096d80 2024-09-13T17:44:19.333 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:44:19.333 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:44:08.385597+0000","modified":"2024-09-13T17:44:11.073569+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5078},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5078":{"gid":5078,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6817/2209249660","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":2209249660}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-09-13T17:44:19.334 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 11 2024-09-13T17:44:19.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.333+0000 7f9ae408b640 1 -- 172.21.15.102:0/1326910663 >> v1:172.21.15.102:6816/34937 conn(0x7f9ac8031160 legacy=0x7f9ac8033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:19.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.333+0000 7f9ae408b640 1 -- 172.21.15.102:0/1326910663 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9adc096d80 msgr2=0x7f9adc13a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:19.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.333+0000 7f9ae408b640 1 --2- 172.21.15.102:0/1326910663 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9adc096d80 0x7f9adc13a7e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9ad400bb80 tx=0x7f9ad4039000 comp rx=0 tx=0).stop 2024-09-13T17:44:19.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.333+0000 7f9ae408b640 1 -- 172.21.15.102:0/1326910663 shutdown_connections 2024-09-13T17:44:19.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.333+0000 7f9ae408b640 1 --2- 172.21.15.102:0/1326910663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9adc12c6a0 0x7f9adc115360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.333+0000 7f9ae408b640 1 --2- 172.21.15.102:0/1326910663 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f9adc09abb0 0x7f9adc13ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.333+0000 7f9ae408b640 1 --2- 172.21.15.102:0/1326910663 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f9adc096d80 0x7f9adc13a7e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.333+0000 7f9ae408b640 1 -- 172.21.15.102:0/1326910663 >> 172.21.15.102:0/1326910663 conn(0x7f9adc09e820 msgr2=0x7f9adc12a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:19.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.333+0000 7f9ae408b640 1 -- 172.21.15.102:0/1326910663 shutdown_connections 2024-09-13T17:44:19.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.333+0000 7f9ae408b640 1 -- 172.21.15.102:0/1326910663 wait complete. 2024-09-13T17:44:19.345 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:44:19.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.436+0000 7fc896754640 1 Processor -- start 2024-09-13T17:44:19.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.436+0000 7fc896754640 1 -- start start 2024-09-13T17:44:19.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.436+0000 7fc896754640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc89009abb0 0x7fc89012c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:19.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.436+0000 7fc896754640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc89012c6a0 0x7fc89012cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:19.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.436+0000 7fc896754640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc89006a530 con 0x7fc890096d80 2024-09-13T17:44:19.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.436+0000 7fc896754640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89009af80 con 0x7fc89009abb0 2024-09-13T17:44:19.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.436+0000 7fc896754640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc89012cff0 con 0x7fc89012c6a0 2024-09-13T17:44:19.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.436+0000 7fc88ffff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc89009abb0 0x7fc89012c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:19.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.436+0000 7fc88f7fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc89012c6a0 0x7fc89012cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:19.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.436+0000 7fc88ffff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc89009abb0 0x7fc89012c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35458/0 (socket says 172.21.15.102:35458) 2024-09-13T17:44:19.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.436+0000 7fc894cf1640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fc890096d80 0x7fc890097220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57322/0 (socket says 172.21.15.102:57322) 2024-09-13T17:44:19.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.436+0000 7fc88f7fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc89012c6a0 0x7fc89012cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:58058/0 (socket says 172.21.15.102:58058) 2024-09-13T17:44:19.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.437+0000 7fc88ffff640 1 -- 172.21.15.102:0/864123760 learned_addr learned my addr 172.21.15.102:0/864123760 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:19.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.437+0000 7fc88ffff640 1 -- 172.21.15.102:0/864123760 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc89012c6a0 msgr2=0x7fc89012cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:19.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.437+0000 7fc88ffff640 1 --2- 172.21.15.102:0/864123760 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc89012c6a0 0x7fc89012cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.437+0000 7fc88ffff640 1 -- 172.21.15.102:0/864123760 >> v1:172.21.15.102:6789/0 conn(0x7fc890096d80 legacy=0x7fc890097220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:19.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.437+0000 7fc88ffff640 1 -- 172.21.15.102:0/864123760 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc890098be0 con 0x7fc89009abb0 2024-09-13T17:44:19.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.437+0000 7fc88effd640 1 -- 172.21.15.102:0/864123760 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2710448867 0 0) 0x7fc89006a530 con 0x7fc890096d80 2024-09-13T17:44:19.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.437+0000 7fc88f7fe640 1 --2- 172.21.15.102:0/864123760 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc89012c6a0 0x7fc89012cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:19.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.437+0000 7fc88ffff640 1 --2- 172.21.15.102:0/864123760 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc89009abb0 0x7fc89012c160 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc880009970 tx=0x7fc8800091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=942f789720708431 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:19.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.438+0000 7fc88effd640 1 -- 172.21.15.102:0/864123760 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88002ad00 con 0x7fc89009abb0 2024-09-13T17:44:19.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.438+0000 7fc88effd640 1 -- 172.21.15.102:0/864123760 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88002ae60 con 0x7fc89009abb0 2024-09-13T17:44:19.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.438+0000 7fc88effd640 1 -- 172.21.15.102:0/864123760 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc880032670 con 0x7fc89009abb0 2024-09-13T17:44:19.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.438+0000 7fc896754640 1 -- 172.21.15.102:0/864123760 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc89009abb0 msgr2=0x7fc89012c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:19.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.438+0000 7fc896754640 1 --2- 172.21.15.102:0/864123760 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc89009abb0 0x7fc89012c160 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc880009970 tx=0x7fc8800091d0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.438+0000 7fc896754640 1 -- 172.21.15.102:0/864123760 shutdown_connections 2024-09-13T17:44:19.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.438+0000 7fc896754640 1 --2- 172.21.15.102:0/864123760 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc89012c6a0 0x7fc89012cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.438+0000 7fc896754640 1 --2- 172.21.15.102:0/864123760 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc89009abb0 0x7fc89012c160 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.438+0000 7fc896754640 1 -- 172.21.15.102:0/864123760 >> 172.21.15.102:0/864123760 conn(0x7fc89009e820 msgr2=0x7fc89009ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:19.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.438+0000 7fc896754640 1 -- 172.21.15.102:0/864123760 shutdown_connections 2024-09-13T17:44:19.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.438+0000 7fc896754640 1 -- 172.21.15.102:0/864123760 wait complete. 2024-09-13T17:44:19.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.439+0000 7fc896754640 1 Processor -- start 2024-09-13T17:44:19.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.439+0000 7fc896754640 1 -- start start 2024-09-13T17:44:19.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.440+0000 7fc896754640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc890096d80 0x7fc8901193c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:19.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.440+0000 7fc896754640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc89009abb0 0x7fc890115be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:19.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.440+0000 7fc896754640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc89012c6a0 0x7fc890116120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:19.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.440+0000 7fc896754640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8901159b0 con 0x7fc890096d80 2024-09-13T17:44:19.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.440+0000 7fc896754640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc890117130 con 0x7fc89012c6a0 2024-09-13T17:44:19.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.440+0000 7fc896754640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc89012a380 con 0x7fc89009abb0 2024-09-13T17:44:19.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.440+0000 7fc88ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc890096d80 0x7fc8901193c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:19.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.440+0000 7fc88f7fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc89009abb0 0x7fc890115be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:19.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.440+0000 7fc88ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc890096d80 0x7fc8901193c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57250/0 (socket says 172.21.15.102:57250) 2024-09-13T17:44:19.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.440+0000 7fc88ffff640 1 -- 172.21.15.102:0/3869650245 learned_addr learned my addr 172.21.15.102:0/3869650245 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:19.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.440+0000 7fc88f7fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc89009abb0 0x7fc890115be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:58068/0 (socket says 172.21.15.102:58068) 2024-09-13T17:44:19.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.440+0000 7fc894cf1640 1 --2- 172.21.15.102:0/3869650245 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc89012c6a0 0x7fc890116120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:19.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.440+0000 7fc88f7fe640 1 -- 172.21.15.102:0/3869650245 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc89012c6a0 msgr2=0x7fc890116120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:19.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.440+0000 7fc88f7fe640 1 --2- 172.21.15.102:0/3869650245 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc89012c6a0 0x7fc890116120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.440+0000 7fc88f7fe640 1 -- 172.21.15.102:0/3869650245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc890096d80 msgr2=0x7fc8901193c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:19.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.440+0000 7fc88f7fe640 1 --2- 172.21.15.102:0/3869650245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc890096d80 0x7fc8901193c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.440+0000 7fc88f7fe640 1 -- 172.21.15.102:0/3869650245 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc884009390 con 0x7fc89009abb0 2024-09-13T17:44:19.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.440+0000 7fc894cf1640 1 --2- 172.21.15.102:0/3869650245 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc89012c6a0 0x7fc890116120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:19.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.440+0000 7fc88ffff640 1 --2- 172.21.15.102:0/3869650245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc890096d80 0x7fc8901193c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:19.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.440+0000 7fc88f7fe640 1 --2- 172.21.15.102:0/3869650245 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc89009abb0 0x7fc890115be0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc884010d00 tx=0x7fc8840090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:19.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.440+0000 7fc877fff640 1 -- 172.21.15.102:0/3869650245 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc884015070 con 0x7fc89009abb0 2024-09-13T17:44:19.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.440+0000 7fc877fff640 1 -- 172.21.15.102:0/3869650245 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc884010e60 con 0x7fc89009abb0 2024-09-13T17:44:19.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.441+0000 7fc896754640 1 -- 172.21.15.102:0/3869650245 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc880009450 con 0x7fc89009abb0 2024-09-13T17:44:19.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.441+0000 7fc877fff640 1 -- 172.21.15.102:0/3869650245 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc884019410 con 0x7fc89009abb0 2024-09-13T17:44:19.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.441+0000 7fc896754640 1 -- 172.21.15.102:0/3869650245 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc890116660 con 0x7fc89009abb0 2024-09-13T17:44:19.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.441+0000 7fc877fff640 1 -- 172.21.15.102:0/3869650245 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fc8840197a0 con 0x7fc89009abb0 2024-09-13T17:44:19.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.442+0000 7fc877fff640 1 -- 172.21.15.102:0/3869650245 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3660+0+0 (secure 0 0 0) 0x7fc884045fa0 con 0x7fc89009abb0 2024-09-13T17:44:19.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.442+0000 7fc896754640 1 -- 172.21.15.102:0/3869650245 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc890098e00 con 0x7fc89009abb0 2024-09-13T17:44:19.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.446+0000 7fc877fff640 1 -- 172.21.15.102:0/3869650245 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fc884022400 con 0x7fc89009abb0 2024-09-13T17:44:19.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.628+0000 7fc896754640 1 -- 172.21.15.102:0/3869650245 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc890116940 con 0x7fc89009abb0 2024-09-13T17:44:19.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.629+0000 7fc877fff640 1 -- 172.21.15.102:0/3869650245 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2268 (secure 0 0 0) 0x7fc884020770 con 0x7fc89009abb0 2024-09-13T17:44:19.631 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:44:19.631 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:44:08.385597+0000","modified":"2024-09-13T17:44:11.073569+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5078},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5078":{"gid":5078,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6817/2209249660","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":2209249660}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-09-13T17:44:19.631 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 11 2024-09-13T17:44:19.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.631+0000 7fc896754640 1 -- 172.21.15.102:0/3869650245 >> v1:172.21.15.102:6816/34937 conn(0x7fc87c031160 legacy=0x7fc87c033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:19.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.631+0000 7fc896754640 1 -- 172.21.15.102:0/3869650245 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc89009abb0 msgr2=0x7fc890115be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:19.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.631+0000 7fc896754640 1 --2- 172.21.15.102:0/3869650245 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc89009abb0 0x7fc890115be0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc884010d00 tx=0x7fc8840090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.631+0000 7fc896754640 1 -- 172.21.15.102:0/3869650245 shutdown_connections 2024-09-13T17:44:19.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.631+0000 7fc896754640 1 --2- 172.21.15.102:0/3869650245 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fc89012c6a0 0x7fc890116120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.631+0000 7fc896754640 1 --2- 172.21.15.102:0/3869650245 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fc89009abb0 0x7fc890115be0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.631+0000 7fc896754640 1 --2- 172.21.15.102:0/3869650245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc890096d80 0x7fc8901193c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.631+0000 7fc896754640 1 -- 172.21.15.102:0/3869650245 >> 172.21.15.102:0/3869650245 conn(0x7fc89009e820 msgr2=0x7fc89012f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:19.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.631+0000 7fc896754640 1 -- 172.21.15.102:0/3869650245 shutdown_connections 2024-09-13T17:44:19.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.631+0000 7fc896754640 1 -- 172.21.15.102:0/3869650245 wait complete. 2024-09-13T17:44:19.644 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:44:19.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.736+0000 7feac944f640 1 Processor -- start 2024-09-13T17:44:19.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.736+0000 7feac944f640 1 -- start start 2024-09-13T17:44:19.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.737+0000 7feac944f640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feac406b140 0x7feac40998c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:19.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.737+0000 7feac944f640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feac409ac20 0x7feac4099e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:19.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.737+0000 7feac944f640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7feac409a340 0x7feac412d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:19.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.737+0000 7feac944f640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac406a680 con 0x7feac409a340 2024-09-13T17:44:19.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.737+0000 7feac944f640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac406ee90 con 0x7feac406b140 2024-09-13T17:44:19.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.737+0000 7feac944f640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feac406ac90 con 0x7feac409ac20 2024-09-13T17:44:19.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.737+0000 7feac2ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feac406b140 0x7feac40998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:19.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.737+0000 7feac37fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7feac409a340 0x7feac412d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:19.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.737+0000 7feaba7fc640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feac409ac20 0x7feac4099e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:19.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.737+0000 7feac2ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feac406b140 0x7feac40998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:35472/0 (socket says 172.21.15.102:35472) 2024-09-13T17:44:19.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.737+0000 7feac2ffd640 1 -- 172.21.15.102:0/790677686 learned_addr learned my addr 172.21.15.102:0/790677686 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:19.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.737+0000 7feac2ffd640 1 -- 172.21.15.102:0/790677686 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feac409ac20 msgr2=0x7feac4099e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:19.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.737+0000 7feac2ffd640 1 --2- 172.21.15.102:0/790677686 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feac409ac20 0x7feac4099e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.737+0000 7feac2ffd640 1 -- 172.21.15.102:0/790677686 >> v2:172.21.15.102:3300/0 conn(0x7feac409a340 msgr2=0x7feac412d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:19.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.737+0000 7feac2ffd640 1 --2- 172.21.15.102:0/790677686 >> v2:172.21.15.102:3300/0 conn(0x7feac409a340 0x7feac412d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.737+0000 7feac2ffd640 1 -- 172.21.15.102:0/790677686 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac4099510 con 0x7feac406b140 2024-09-13T17:44:19.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.737+0000 7feac37fe640 1 --2- 172.21.15.102:0/790677686 >> v2:172.21.15.102:3300/0 conn(0x7feac409a340 0x7feac412d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:19.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.737+0000 7feaba7fc640 1 --2- 172.21.15.102:0/790677686 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feac409ac20 0x7feac4099e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:19.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.738+0000 7feac2ffd640 1 --2- 172.21.15.102:0/790677686 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feac406b140 0x7feac40998c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7feab4009910 tx=0x7feab40091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98998402aaf0def server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:19.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.738+0000 7feac27fc640 1 -- 172.21.15.102:0/790677686 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab402ac80 con 0x7feac406b140 2024-09-13T17:44:19.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.738+0000 7feac27fc640 1 -- 172.21.15.102:0/790677686 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab402ade0 con 0x7feac406b140 2024-09-13T17:44:19.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.738+0000 7feac27fc640 1 -- 172.21.15.102:0/790677686 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab4032670 con 0x7feac406b140 2024-09-13T17:44:19.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.738+0000 7feac944f640 1 -- 172.21.15.102:0/790677686 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feac406b140 msgr2=0x7feac40998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:19.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.738+0000 7feac944f640 1 --2- 172.21.15.102:0/790677686 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feac406b140 0x7feac40998c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7feab4009910 tx=0x7feab40091d0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.739+0000 7feac944f640 1 -- 172.21.15.102:0/790677686 shutdown_connections 2024-09-13T17:44:19.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.739+0000 7feac944f640 1 --2- 172.21.15.102:0/790677686 >> v2:172.21.15.102:3300/0 conn(0x7feac409a340 0x7feac412d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.739+0000 7feac944f640 1 --2- 172.21.15.102:0/790677686 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feac409ac20 0x7feac4099e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.739+0000 7feac944f640 1 --2- 172.21.15.102:0/790677686 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feac406b140 0x7feac40998c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.739+0000 7feac944f640 1 -- 172.21.15.102:0/790677686 >> 172.21.15.102:0/790677686 conn(0x7feac409e820 msgr2=0x7feac409ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:19.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.739+0000 7feac944f640 1 -- 172.21.15.102:0/790677686 shutdown_connections 2024-09-13T17:44:19.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.739+0000 7feac944f640 1 -- 172.21.15.102:0/790677686 wait complete. 2024-09-13T17:44:19.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.739+0000 7feac944f640 1 Processor -- start 2024-09-13T17:44:19.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.740+0000 7feac944f640 1 -- start start 2024-09-13T17:44:19.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.740+0000 7feac944f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feac406b140 0x7feac411f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:19.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.740+0000 7feac944f640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feac409a340 0x7feac412c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:19.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.740+0000 7feac944f640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feac409ac20 0x7feac411b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:19.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.740+0000 7feac944f640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac411ebe0 con 0x7feac406b140 2024-09-13T17:44:19.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.740+0000 7feac944f640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac411f180 con 0x7feac409ac20 2024-09-13T17:44:19.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.740+0000 7feac944f640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feac406b8a0 con 0x7feac409a340 2024-09-13T17:44:19.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.740+0000 7feac2ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feac406b140 0x7feac411f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:19.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.740+0000 7feaba7fc640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feac409a340 0x7feac412c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:19.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.740+0000 7feac2ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feac406b140 0x7feac411f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57276/0 (socket says 172.21.15.102:57276) 2024-09-13T17:44:19.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.740+0000 7feac2ffd640 1 -- 172.21.15.102:0/3993426513 learned_addr learned my addr 172.21.15.102:0/3993426513 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:19.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.740+0000 7feac37fe640 1 --2- 172.21.15.102:0/3993426513 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feac409ac20 0x7feac411b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:19.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.740+0000 7feac2ffd640 1 -- 172.21.15.102:0/3993426513 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feac409a340 msgr2=0x7feac412c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:19.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.740+0000 7feac2ffd640 1 --2- 172.21.15.102:0/3993426513 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feac409a340 0x7feac412c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.740+0000 7feac2ffd640 1 -- 172.21.15.102:0/3993426513 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feac409ac20 msgr2=0x7feac411b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:19.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.740+0000 7feac2ffd640 1 --2- 172.21.15.102:0/3993426513 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feac409ac20 0x7feac411b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.740+0000 7feac2ffd640 1 -- 172.21.15.102:0/3993426513 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feabc009a20 con 0x7feac406b140 2024-09-13T17:44:19.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.740+0000 7feac37fe640 1 --2- 172.21.15.102:0/3993426513 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feac409ac20 0x7feac411b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:19.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.740+0000 7feaba7fc640 1 --2- 172.21.15.102:0/3993426513 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feac409a340 0x7feac412c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:19.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.741+0000 7feac2ffd640 1 --2- 172.21.15.102:0/3993426513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feac406b140 0x7feac411f680 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7feab402a5b0 tx=0x7feab4032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:19.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.741+0000 7feabb7fe640 1 -- 172.21.15.102:0/3993426513 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab403d630 con 0x7feac406b140 2024-09-13T17:44:19.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.741+0000 7feabb7fe640 1 -- 172.21.15.102:0/3993426513 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab403d7b0 con 0x7feac406b140 2024-09-13T17:44:19.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.741+0000 7feac944f640 1 -- 172.21.15.102:0/3993426513 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaac009390 con 0x7feac406b140 2024-09-13T17:44:19.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.741+0000 7feabb7fe640 1 -- 172.21.15.102:0/3993426513 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab403db70 con 0x7feac406b140 2024-09-13T17:44:19.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.741+0000 7feac944f640 1 -- 172.21.15.102:0/3993426513 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab4009450 con 0x7feac406b140 2024-09-13T17:44:19.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.742+0000 7feabb7fe640 1 -- 172.21.15.102:0/3993426513 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7feab4039040 con 0x7feac406b140 2024-09-13T17:44:19.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.742+0000 7feabb7fe640 1 -- 172.21.15.102:0/3993426513 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3660+0+0 (secure 0 0 0) 0x7feab40619c0 con 0x7feac406b140 2024-09-13T17:44:19.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.742+0000 7feac944f640 1 -- 172.21.15.102:0/3993426513 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feac4064000 con 0x7feac406b140 2024-09-13T17:44:19.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.746+0000 7feabb7fe640 1 -- 172.21.15.102:0/3993426513 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7feab4030340 con 0x7feac406b140 2024-09-13T17:44:19.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.899+0000 7feac944f640 1 -- 172.21.15.102:0/3993426513 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feac4067690 con 0x7feac406b140 2024-09-13T17:44:19.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.900+0000 7feabb7fe640 1 -- 172.21.15.102:0/3993426513 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+11330 (secure 0 0 0) 0x7feab4042020 con 0x7feac406b140 2024-09-13T17:44:19.902 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:44:19.902 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":32,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:44:10.068600+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:44:06.718328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-13T17:44:07.355624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:44:19.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.902+0000 7feac944f640 1 -- 172.21.15.102:0/3993426513 >> v1:172.21.15.102:6816/34937 conn(0x7feaa4031160 legacy=0x7feaa4033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:19.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.902+0000 7feac944f640 1 -- 172.21.15.102:0/3993426513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feac406b140 msgr2=0x7feac411f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:19.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.902+0000 7feac944f640 1 --2- 172.21.15.102:0/3993426513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feac406b140 0x7feac411f680 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7feab402a5b0 tx=0x7feab4032000 comp rx=0 tx=0).stop 2024-09-13T17:44:19.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.902+0000 7feac944f640 1 -- 172.21.15.102:0/3993426513 shutdown_connections 2024-09-13T17:44:19.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.902+0000 7feac944f640 1 --2- 172.21.15.102:0/3993426513 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feac409ac20 0x7feac411b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.902+0000 7feac944f640 1 --2- 172.21.15.102:0/3993426513 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feac409a340 0x7feac412c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.902+0000 7feac944f640 1 --2- 172.21.15.102:0/3993426513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feac406b140 0x7feac411f680 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.902+0000 7feac944f640 1 -- 172.21.15.102:0/3993426513 >> 172.21.15.102:0/3993426513 conn(0x7feac409e820 msgr2=0x7feac406e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:19.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.902+0000 7feac944f640 1 -- 172.21.15.102:0/3993426513 shutdown_connections 2024-09-13T17:44:19.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:19.902+0000 7feac944f640 1 -- 172.21.15.102:0/3993426513 wait complete. 2024-09-13T17:44:19.914 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-13T17:44:20.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.007+0000 7f32b6954640 1 Processor -- start 2024-09-13T17:44:20.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.007+0000 7f32b6954640 1 -- start start 2024-09-13T17:44:20.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.008+0000 7f32b6954640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32b0128330 0x7f32b0128700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:20.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.008+0000 7f32b6954640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f32b0128c40 0x7f32b012b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:20.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.008+0000 7f32b6954640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b0097be0 con 0x7f32b00991b0 2024-09-13T17:44:20.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.008+0000 7f32b6954640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32b0098e70 con 0x7f32b0128c40 2024-09-13T17:44:20.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.008+0000 7f32b6954640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32b0098310 con 0x7f32b0128330 2024-09-13T17:44:20.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.008+0000 7f32affff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32b0128330 0x7f32b0128700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:20.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.008+0000 7f32af7fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f32b0128c40 0x7f32b012b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:20.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.008+0000 7f32b4ef1640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f32b00991b0 0x7f32b012bd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57334/0 (socket says 172.21.15.102:57334) 2024-09-13T17:44:20.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.008+0000 7f32b4ef1640 1 -- 172.21.15.102:0/3351501446 learned_addr learned my addr 172.21.15.102:0/3351501446 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:44:20.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.008+0000 7f32af7fe640 1 -- 172.21.15.102:0/3351501446 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32b0128330 msgr2=0x7f32b0128700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:20.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.008+0000 7f32af7fe640 1 --2- 172.21.15.102:0/3351501446 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32b0128330 0x7f32b0128700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:20.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.008+0000 7f32af7fe640 1 -- 172.21.15.102:0/3351501446 >> v1:172.21.15.102:6789/0 conn(0x7f32b00991b0 legacy=0x7f32b012bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:20.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.008+0000 7f32af7fe640 1 -- 172.21.15.102:0/3351501446 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b00985d0 con 0x7f32b0128c40 2024-09-13T17:44:20.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.008+0000 7f32affff640 1 --2- 172.21.15.102:0/3351501446 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32b0128330 0x7f32b0128700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:20.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.009+0000 7f32af7fe640 1 --2- 172.21.15.102:0/3351501446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b0128c40 0x7f32b012b840 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f32a4009850 tx=0x7f32a40090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c026b591da1d4625 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:20.015 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.009+0000 7f32aeffd640 1 -- 172.21.15.102:0/3351501446 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a402ad90 con 0x7f32b0128c40 2024-09-13T17:44:20.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.009+0000 7f32aeffd640 1 -- 172.21.15.102:0/3351501446 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a4032440 con 0x7f32b0128c40 2024-09-13T17:44:20.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.009+0000 7f32aeffd640 1 -- 172.21.15.102:0/3351501446 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a4032820 con 0x7f32b0128c40 2024-09-13T17:44:20.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.009+0000 7f32b6954640 1 -- 172.21.15.102:0/3351501446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b0128c40 msgr2=0x7f32b012b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:20.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.009+0000 7f32b6954640 1 --2- 172.21.15.102:0/3351501446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b0128c40 0x7f32b012b840 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f32a4009850 tx=0x7f32a40090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:20.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.010+0000 7f32b6954640 1 -- 172.21.15.102:0/3351501446 shutdown_connections 2024-09-13T17:44:20.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.010+0000 7f32b6954640 1 --2- 172.21.15.102:0/3351501446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b0128c40 0x7f32b012b840 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:20.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.010+0000 7f32b6954640 1 --2- 172.21.15.102:0/3351501446 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32b0128330 0x7f32b0128700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:20.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.010+0000 7f32b6954640 1 -- 172.21.15.102:0/3351501446 >> 172.21.15.102:0/3351501446 conn(0x7f32b009e820 msgr2=0x7f32b009ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:20.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.010+0000 7f32b6954640 1 -- 172.21.15.102:0/3351501446 shutdown_connections 2024-09-13T17:44:20.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.010+0000 7f32b6954640 1 -- 172.21.15.102:0/3351501446 wait complete. 2024-09-13T17:44:20.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.010+0000 7f32b6954640 1 Processor -- start 2024-09-13T17:44:20.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.010+0000 7f32b6954640 1 -- start start 2024-09-13T17:44:20.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.011+0000 7f32b6954640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32b00991b0 0x7f32b013ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:20.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.011+0000 7f32b6954640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b0128330 0x7f32b013d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:20.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.011+0000 7f32b6954640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32b0128c40 0x7f32b006c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:20.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.011+0000 7f32b6954640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b0130a60 con 0x7f32b0128330 2024-09-13T17:44:20.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.011+0000 7f32b6954640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b0115630 con 0x7f32b00991b0 2024-09-13T17:44:20.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.011+0000 7f32b6954640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32b006f230 con 0x7f32b0128c40 2024-09-13T17:44:20.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.011+0000 7f32af7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b0128330 0x7f32b013d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:20.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.011+0000 7f32affff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32b00991b0 0x7f32b013ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:20.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.011+0000 7f32b4ef1640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32b0128c40 0x7f32b006c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:20.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.011+0000 7f32af7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b0128330 0x7f32b013d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57288/0 (socket says 172.21.15.102:57288) 2024-09-13T17:44:20.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.011+0000 7f32af7fe640 1 -- 172.21.15.102:0/3549173069 learned_addr learned my addr 172.21.15.102:0/3549173069 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:20.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.011+0000 7f32affff640 1 -- 172.21.15.102:0/3549173069 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32b0128c40 msgr2=0x7f32b006c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:20.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.011+0000 7f32affff640 1 --2- 172.21.15.102:0/3549173069 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32b0128c40 0x7f32b006c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:20.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.011+0000 7f32affff640 1 -- 172.21.15.102:0/3549173069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b0128330 msgr2=0x7f32b013d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:20.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.011+0000 7f32affff640 1 --2- 172.21.15.102:0/3549173069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b0128330 0x7f32b013d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:20.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.011+0000 7f32affff640 1 -- 172.21.15.102:0/3549173069 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a0009390 con 0x7f32b00991b0 2024-09-13T17:44:20.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.011+0000 7f32af7fe640 1 --2- 172.21.15.102:0/3549173069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b0128330 0x7f32b013d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:20.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.011+0000 7f32b4ef1640 1 --2- 172.21.15.102:0/3549173069 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32b0128c40 0x7f32b006c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:20.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.011+0000 7f32affff640 1 --2- 172.21.15.102:0/3549173069 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32b00991b0 0x7f32b013ccb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f32a000ea50 tx=0x7f32a00090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:20.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.012+0000 7f3297fff640 1 -- 172.21.15.102:0/3549173069 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a0009670 con 0x7f32b00991b0 2024-09-13T17:44:20.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.012+0000 7f3297fff640 1 -- 172.21.15.102:0/3549173069 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a000ca10 con 0x7f32b00991b0 2024-09-13T17:44:20.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.012+0000 7f32b6954640 1 -- 172.21.15.102:0/3549173069 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32a4009390 con 0x7f32b00991b0 2024-09-13T17:44:20.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.012+0000 7f3297fff640 1 -- 172.21.15.102:0/3549173069 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a000cdb0 con 0x7f32b00991b0 2024-09-13T17:44:20.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.012+0000 7f32b6954640 1 -- 172.21.15.102:0/3549173069 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b0125d60 con 0x7f32b00991b0 2024-09-13T17:44:20.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.013+0000 7f3297fff640 1 -- 172.21.15.102:0/3549173069 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f32a0013680 con 0x7f32b00991b0 2024-09-13T17:44:20.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.013+0000 7f3297fff640 1 -- 172.21.15.102:0/3549173069 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3660+0+0 (secure 0 0 0) 0x7f32a0042320 con 0x7f32b00991b0 2024-09-13T17:44:20.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.013+0000 7f32b6954640 1 -- 172.21.15.102:0/3549173069 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b01159d0 con 0x7f32b00991b0 2024-09-13T17:44:20.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.017+0000 7f3297fff640 1 -- 172.21.15.102:0/3549173069 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f32a0021020 con 0x7f32b00991b0 2024-09-13T17:44:20.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.202+0000 7f32b6954640 1 -- 172.21.15.102:0/3549173069 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f32b0064000 con 0x7f32b00991b0 2024-09-13T17:44:20.204 INFO:tasks.ceph.mon.a.smithi102.stderr:2024-09-13T17:44:20.203+0000 7fa5873f1640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T17:44:20.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.208+0000 7f3297fff640 1 -- 172.21.15.102:0/3549173069 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v12) v1 ==== 161+0+0 (secure 0 0 0) 0x7f32a00236b0 con 0x7f32b00991b0 2024-09-13T17:44:20.210 INFO:teuthology.orchestra.run.smithi102.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-13T17:44:20.211 INFO:tasks.ceph.mds.a.smithi102.stderr: -1086> 2024-09-13T17:44:12.040+0000 7f1c34ff1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:44:20.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.211+0000 7f32b6954640 1 -- 172.21.15.102:0/3549173069 >> v1:172.21.15.102:6816/34937 conn(0x7f329c031160 legacy=0x7f329c033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:20.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.211+0000 7f32b6954640 1 -- 172.21.15.102:0/3549173069 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32b00991b0 msgr2=0x7f32b013ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:20.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.211+0000 7f32b6954640 1 --2- 172.21.15.102:0/3549173069 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32b00991b0 0x7f32b013ccb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f32a000ea50 tx=0x7f32a00090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:20.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.211+0000 7f32b6954640 1 -- 172.21.15.102:0/3549173069 shutdown_connections 2024-09-13T17:44:20.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.211+0000 7f32b6954640 1 --2- 172.21.15.102:0/3549173069 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32b0128c40 0x7f32b006c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:20.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.211+0000 7f32b6954640 1 --2- 172.21.15.102:0/3549173069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b0128330 0x7f32b013d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:20.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.211+0000 7f32b6954640 1 --2- 172.21.15.102:0/3549173069 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32b00991b0 0x7f32b013ccb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:20.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.211+0000 7f32b6954640 1 -- 172.21.15.102:0/3549173069 >> 172.21.15.102:0/3549173069 conn(0x7f32b009e820 msgr2=0x7f32b0126920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:20.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.211+0000 7f32b6954640 1 -- 172.21.15.102:0/3549173069 shutdown_connections 2024-09-13T17:44:20.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.211+0000 7f32b6954640 1 -- 172.21.15.102:0/3549173069 wait complete. 2024-09-13T17:44:20.224 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-13T17:44:20.233 INFO:tasks.ceph.mds.a.smithi102.stdout:starting mds.a at 2024-09-13T17:44:20.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.317+0000 7f7a36478640 1 Processor -- start 2024-09-13T17:44:20.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.317+0000 7f7a36478640 1 -- start start 2024-09-13T17:44:20.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.318+0000 7f7a36478640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7a3009db30 0x7f7a30130b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:20.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.318+0000 7f7a36478640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f7a30131060 0x7f7a30133440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:20.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.318+0000 7f7a36478640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a30098e00 con 0x7f7a3009d050 2024-09-13T17:44:20.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.318+0000 7f7a36478640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a3009df20 con 0x7f7a30131060 2024-09-13T17:44:20.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.318+0000 7f7a36478640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7a30068b60 con 0x7f7a3009db30 2024-09-13T17:44:20.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.318+0000 7f7a2f7fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7a3009db30 0x7f7a30130b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:20.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.318+0000 7f7a34a15640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f7a30131060 0x7f7a30133440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:20.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.318+0000 7f7a2ffff640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f7a3009d050 0x7f7a3009d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57356/0 (socket says 172.21.15.102:57356) 2024-09-13T17:44:20.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.318+0000 7f7a2ffff640 1 -- 172.21.15.102:0/672085633 learned_addr learned my addr 172.21.15.102:0/672085633 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:44:20.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.318+0000 7f7a34a15640 1 -- 172.21.15.102:0/672085633 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7a3009db30 msgr2=0x7f7a30130b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:20.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.318+0000 7f7a34a15640 1 --2- 172.21.15.102:0/672085633 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7a3009db30 0x7f7a30130b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:20.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.318+0000 7f7a34a15640 1 -- 172.21.15.102:0/672085633 >> v1:172.21.15.102:6789/0 conn(0x7f7a3009d050 legacy=0x7f7a3009d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:20.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.318+0000 7f7a34a15640 1 -- 172.21.15.102:0/672085633 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a30096fd0 con 0x7f7a30131060 2024-09-13T17:44:20.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.318+0000 7f7a2f7fe640 1 --2- 172.21.15.102:0/672085633 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7a3009db30 0x7f7a30130b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:20.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.319+0000 7f7a34a15640 1 --2- 172.21.15.102:0/672085633 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a30131060 0x7f7a30133440 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7a28009e60 tx=0x7f7a28009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58734cd911a8a30d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:20.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.319+0000 7f7a2effd640 1 -- 172.21.15.102:0/672085633 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a28007550 con 0x7f7a30131060 2024-09-13T17:44:20.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.319+0000 7f7a2effd640 1 -- 172.21.15.102:0/672085633 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2802d050 con 0x7f7a30131060 2024-09-13T17:44:20.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.319+0000 7f7a2effd640 1 -- 172.21.15.102:0/672085633 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a280078e0 con 0x7f7a30131060 2024-09-13T17:44:20.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.320+0000 7f7a36478640 1 -- 172.21.15.102:0/672085633 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a30131060 msgr2=0x7f7a30133440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:20.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.320+0000 7f7a36478640 1 --2- 172.21.15.102:0/672085633 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a30131060 0x7f7a30133440 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7a28009e60 tx=0x7f7a28009780 comp rx=0 tx=0).stop 2024-09-13T17:44:20.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.320+0000 7f7a36478640 1 -- 172.21.15.102:0/672085633 shutdown_connections 2024-09-13T17:44:20.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.320+0000 7f7a36478640 1 --2- 172.21.15.102:0/672085633 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a30131060 0x7f7a30133440 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:20.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.320+0000 7f7a36478640 1 --2- 172.21.15.102:0/672085633 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7a3009db30 0x7f7a30130b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:20.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.320+0000 7f7a36478640 1 -- 172.21.15.102:0/672085633 >> 172.21.15.102:0/672085633 conn(0x7f7a3009e8c0 msgr2=0x7f7a3009ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:20.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.320+0000 7f7a36478640 1 -- 172.21.15.102:0/672085633 shutdown_connections 2024-09-13T17:44:20.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.320+0000 7f7a36478640 1 -- 172.21.15.102:0/672085633 wait complete. 2024-09-13T17:44:20.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.320+0000 7f7a36478640 1 Processor -- start 2024-09-13T17:44:20.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.321+0000 7f7a36478640 1 -- start start 2024-09-13T17:44:20.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.321+0000 7f7a36478640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7a3009d050 0x7f7a3006dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:20.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.321+0000 7f7a36478640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7a3009db30 0x7f7a3006e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:20.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.321+0000 7f7a36478640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a30131060 0x7f7a3012ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:20.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.321+0000 7f7a36478640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3012e2d0 con 0x7f7a30131060 2024-09-13T17:44:20.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.321+0000 7f7a36478640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3006a790 con 0x7f7a3009db30 2024-09-13T17:44:20.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.321+0000 7f7a36478640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3006aa30 con 0x7f7a3009d050 2024-09-13T17:44:20.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.321+0000 7f7a2ffff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7a3009d050 0x7f7a3006dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:20.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.321+0000 7f7a2f7fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7a3009db30 0x7f7a3006e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:20.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.321+0000 7f7a2ffff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7a3009d050 0x7f7a3006dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:58126/0 (socket says 172.21.15.102:58126) 2024-09-13T17:44:20.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.321+0000 7f7a2ffff640 1 -- 172.21.15.102:0/3671262098 learned_addr learned my addr 172.21.15.102:0/3671262098 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:20.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.321+0000 7f7a34a15640 1 --2- 172.21.15.102:0/3671262098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a30131060 0x7f7a3012ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:20.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.321+0000 7f7a2ffff640 1 -- 172.21.15.102:0/3671262098 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7a3009db30 msgr2=0x7f7a3006e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:20.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.321+0000 7f7a2ffff640 1 --2- 172.21.15.102:0/3671262098 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7a3009db30 0x7f7a3006e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:20.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.321+0000 7f7a2ffff640 1 -- 172.21.15.102:0/3671262098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a30131060 msgr2=0x7f7a3012ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:20.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.321+0000 7f7a2ffff640 1 --2- 172.21.15.102:0/3671262098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a30131060 0x7f7a3012ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:20.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.321+0000 7f7a2ffff640 1 -- 172.21.15.102:0/3671262098 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a24009390 con 0x7f7a3009d050 2024-09-13T17:44:20.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.322+0000 7f7a2ffff640 1 --2- 172.21.15.102:0/3671262098 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7a3009d050 0x7f7a3006dc60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000c590 tx=0x7f7a20007ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:20.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.322+0000 7f7a17fff640 1 -- 172.21.15.102:0/3671262098 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a20005aa0 con 0x7f7a3009d050 2024-09-13T17:44:20.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.322+0000 7f7a17fff640 1 -- 172.21.15.102:0/3671262098 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a20005c20 con 0x7f7a3009d050 2024-09-13T17:44:20.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.322+0000 7f7a36478640 1 -- 172.21.15.102:0/3671262098 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a28009a20 con 0x7f7a3009d050 2024-09-13T17:44:20.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.322+0000 7f7a17fff640 1 -- 172.21.15.102:0/3671262098 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a200070f0 con 0x7f7a3009d050 2024-09-13T17:44:20.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.322+0000 7f7a36478640 1 -- 172.21.15.102:0/3671262098 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a3012e9e0 con 0x7f7a3009d050 2024-09-13T17:44:20.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.323+0000 7f7a17fff640 1 -- 172.21.15.102:0/3671262098 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f7a20007540 con 0x7f7a3009d050 2024-09-13T17:44:20.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.323+0000 7f7a17fff640 1 -- 172.21.15.102:0/3671262098 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3703+0+0 (secure 0 0 0) 0x7f7a20041440 con 0x7f7a3009d050 2024-09-13T17:44:20.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.323+0000 7f7a36478640 1 -- 172.21.15.102:0/3671262098 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a3009b500 con 0x7f7a3009d050 2024-09-13T17:44:20.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.327+0000 7f7a17fff640 1 -- 172.21.15.102:0/3671262098 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f7a3009b500 con 0x7f7a3009d050 2024-09-13T17:44:20.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:20.515+0000 7f7a36478640 1 -- 172.21.15.102:0/3671262098 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7a3009b500 con 0x7f7a3009d050 2024-09-13T17:44:21.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.211+0000 7f7a17fff640 1 -- 172.21.15.102:0/3671262098 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7a3009b500 con 0x7f7a3009d050 2024-09-13T17:44:21.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.214+0000 7f7a36478640 1 -- 172.21.15.102:0/3671262098 >> v1:172.21.15.102:6816/34937 conn(0x7f7a1c031160 legacy=0x7f7a1c033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:21.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.214+0000 7f7a36478640 1 -- 172.21.15.102:0/3671262098 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7a3009d050 msgr2=0x7f7a3006dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:21.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.214+0000 7f7a36478640 1 --2- 172.21.15.102:0/3671262098 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7a3009d050 0x7f7a3006dc60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000c590 tx=0x7f7a20007ee0 comp rx=0 tx=0).stop 2024-09-13T17:44:21.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.214+0000 7f7a36478640 1 -- 172.21.15.102:0/3671262098 shutdown_connections 2024-09-13T17:44:21.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.214+0000 7f7a36478640 1 --2- 172.21.15.102:0/3671262098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a30131060 0x7f7a3012ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:21.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.214+0000 7f7a36478640 1 --2- 172.21.15.102:0/3671262098 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f7a3009db30 0x7f7a3006e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:21.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.214+0000 7f7a36478640 1 --2- 172.21.15.102:0/3671262098 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f7a3009d050 0x7f7a3006dc60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:21.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.214+0000 7f7a36478640 1 -- 172.21.15.102:0/3671262098 >> 172.21.15.102:0/3671262098 conn(0x7f7a3009e8c0 msgr2=0x7f7a30133440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:21.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.214+0000 7f7a36478640 1 -- 172.21.15.102:0/3671262098 shutdown_connections 2024-09-13T17:44:21.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.214+0000 7f7a36478640 1 -- 172.21.15.102:0/3671262098 wait complete. 2024-09-13T17:44:21.227 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-13T17:44:21.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.363+0000 7fef5571d640 1 Processor -- start 2024-09-13T17:44:21.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.363+0000 7fef5571d640 1 -- start start 2024-09-13T17:44:21.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.364+0000 7fef5571d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef5006d8f0 0x7fef5009e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:21.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.364+0000 7fef5571d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fef5009ec20 0x7fef5009f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:21.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.364+0000 7fef5571d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef5009f550 con 0x7fef501320d0 2024-09-13T17:44:21.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.364+0000 7fef5571d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef5006c440 con 0x7fef5009ec20 2024-09-13T17:44:21.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.364+0000 7fef5571d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef5009fb50 con 0x7fef5006d8f0 2024-09-13T17:44:21.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.364+0000 7fef4effd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef5006d8f0 0x7fef5009e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:21.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.364+0000 7fef467fc640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fef5009ec20 0x7fef5009f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:21.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.364+0000 7fef4effd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef5006d8f0 0x7fef5009e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:58134/0 (socket says 172.21.15.102:58134) 2024-09-13T17:44:21.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.364+0000 7fef4effd640 1 -- 172.21.15.102:0/3983541946 learned_addr learned my addr 172.21.15.102:0/3983541946 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:21.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.364+0000 7fef467fc640 1 --2- 172.21.15.102:0/3983541946 >> v2:172.21.15.102:3300/0 conn(0x7fef5009ec20 0x7fef5009f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57328/0 (socket says 172.21.15.102:57328) 2024-09-13T17:44:21.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.364+0000 7fef467fc640 1 -- 172.21.15.102:0/3983541946 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef5006d8f0 msgr2=0x7fef5009e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:21.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.364+0000 7fef467fc640 1 --2- 172.21.15.102:0/3983541946 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef5006d8f0 0x7fef5009e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:21.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.364+0000 7fef467fc640 1 -- 172.21.15.102:0/3983541946 >> v1:172.21.15.102:6789/0 conn(0x7fef501320d0 legacy=0x7fef501344b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:21.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.364+0000 7fef467fc640 1 -- 172.21.15.102:0/3983541946 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef5006dcc0 con 0x7fef5009ec20 2024-09-13T17:44:21.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.364+0000 7fef4effd640 1 --2- 172.21.15.102:0/3983541946 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef5006d8f0 0x7fef5009e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:21.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.365+0000 7fef467fc640 1 --2- 172.21.15.102:0/3983541946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef5009ec20 0x7fef5009f010 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fef3800dca0 tx=0x7fef3800d5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6059243f7781d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:21.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.365+0000 7fef4e7fc640 1 -- 172.21.15.102:0/3983541946 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3800d2e0 con 0x7fef5009ec20 2024-09-13T17:44:21.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.365+0000 7fef4e7fc640 1 -- 172.21.15.102:0/3983541946 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef38031050 con 0x7fef5009ec20 2024-09-13T17:44:21.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.365+0000 7fef4e7fc640 1 -- 172.21.15.102:0/3983541946 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3800b680 con 0x7fef5009ec20 2024-09-13T17:44:21.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.366+0000 7fef5571d640 1 -- 172.21.15.102:0/3983541946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef5009ec20 msgr2=0x7fef5009f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:21.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.366+0000 7fef5571d640 1 --2- 172.21.15.102:0/3983541946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef5009ec20 0x7fef5009f010 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fef3800dca0 tx=0x7fef3800d5b0 comp rx=0 tx=0).stop 2024-09-13T17:44:21.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.366+0000 7fef5571d640 1 -- 172.21.15.102:0/3983541946 shutdown_connections 2024-09-13T17:44:21.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.366+0000 7fef5571d640 1 --2- 172.21.15.102:0/3983541946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef5009ec20 0x7fef5009f010 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:21.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.366+0000 7fef5571d640 1 --2- 172.21.15.102:0/3983541946 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef5006d8f0 0x7fef5009e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:21.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.366+0000 7fef5571d640 1 -- 172.21.15.102:0/3983541946 >> 172.21.15.102:0/3983541946 conn(0x7fef50067c00 msgr2=0x7fef500a08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:21.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.366+0000 7fef5571d640 1 -- 172.21.15.102:0/3983541946 shutdown_connections 2024-09-13T17:44:21.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.366+0000 7fef5571d640 1 -- 172.21.15.102:0/3983541946 wait complete. 2024-09-13T17:44:21.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.367+0000 7fef5571d640 1 Processor -- start 2024-09-13T17:44:21.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.367+0000 7fef5571d640 1 -- start start 2024-09-13T17:44:21.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.367+0000 7fef5571d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef5006d8f0 0x7fef5012d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:21.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.367+0000 7fef5571d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef5009ec20 0x7fef50116f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:21.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.367+0000 7fef5571d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fef501320d0 0x7fef50117480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:21.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.367+0000 7fef5571d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef50119f00 con 0x7fef5009ec20 2024-09-13T17:44:21.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.367+0000 7fef5571d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef50118330 con 0x7fef501320d0 2024-09-13T17:44:21.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.367+0000 7fef5571d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef5006c690 con 0x7fef5006d8f0 2024-09-13T17:44:21.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.367+0000 7fef4effd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef5006d8f0 0x7fef5012d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:21.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.367+0000 7fef467fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef5009ec20 0x7fef50116f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:21.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.367+0000 7fef467fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef5009ec20 0x7fef50116f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57332/0 (socket says 172.21.15.102:57332) 2024-09-13T17:44:21.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.367+0000 7fef4effd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef5006d8f0 0x7fef5012d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:58140/0 (socket says 172.21.15.102:58140) 2024-09-13T17:44:21.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.367+0000 7fef467fc640 1 -- 172.21.15.102:0/586480175 learned_addr learned my addr 172.21.15.102:0/586480175 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:21.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.367+0000 7fef4f7fe640 1 --2- 172.21.15.102:0/586480175 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fef501320d0 0x7fef50117480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:21.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.367+0000 7fef467fc640 1 -- 172.21.15.102:0/586480175 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef5006d8f0 msgr2=0x7fef5012d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:21.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.367+0000 7fef467fc640 1 --2- 172.21.15.102:0/586480175 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef5006d8f0 0x7fef5012d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:21.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.367+0000 7fef467fc640 1 -- 172.21.15.102:0/586480175 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fef501320d0 msgr2=0x7fef50117480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:21.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.367+0000 7fef467fc640 1 --2- 172.21.15.102:0/586480175 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fef501320d0 0x7fef50117480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:21.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.367+0000 7fef467fc640 1 -- 172.21.15.102:0/586480175 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef40009450 con 0x7fef5009ec20 2024-09-13T17:44:21.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.368+0000 7fef467fc640 1 --2- 172.21.15.102:0/586480175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef5009ec20 0x7fef50116f40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fef3800dca0 tx=0x7fef3803a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:21.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.368+0000 7fef477fe640 1 -- 172.21.15.102:0/586480175 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef380367f0 con 0x7fef5009ec20 2024-09-13T17:44:21.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.368+0000 7fef477fe640 1 -- 172.21.15.102:0/586480175 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef38036970 con 0x7fef5009ec20 2024-09-13T17:44:21.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.368+0000 7fef477fe640 1 -- 172.21.15.102:0/586480175 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef38036d30 con 0x7fef5009ec20 2024-09-13T17:44:21.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.368+0000 7fef5571d640 1 -- 172.21.15.102:0/586480175 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef3800d830 con 0x7fef5009ec20 2024-09-13T17:44:21.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.368+0000 7fef5571d640 1 -- 172.21.15.102:0/586480175 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef50118540 con 0x7fef5009ec20 2024-09-13T17:44:21.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.368+0000 7fef477fe640 1 -- 172.21.15.102:0/586480175 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fef3803c050 con 0x7fef5009ec20 2024-09-13T17:44:21.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.369+0000 7fef477fe640 1 -- 172.21.15.102:0/586480175 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3703+0+0 (secure 0 0 0) 0x7fef38066820 con 0x7fef5009ec20 2024-09-13T17:44:21.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.369+0000 7fef5571d640 1 -- 172.21.15.102:0/586480175 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef50064600 con 0x7fef5009ec20 2024-09-13T17:44:21.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.373+0000 7fef477fe640 1 -- 172.21.15.102:0/586480175 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fef3804d080 con 0x7fef5009ec20 2024-09-13T17:44:21.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:21.524+0000 7fef5571d640 1 -- 172.21.15.102:0/586480175 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fef5009dbe0 con 0x7fef5009ec20 2024-09-13T17:44:22.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.213+0000 7fef477fe640 1 -- 172.21.15.102:0/586480175 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v34) v1 ==== 181+0+0 (secure 0 0 0) 0x7fef38043020 con 0x7fef5009ec20 2024-09-13T17:44:22.215 INFO:teuthology.orchestra.run.smithi102.stderr:pool 'cephfs_metadata' removed 2024-09-13T17:44:22.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.217+0000 7fef5571d640 1 -- 172.21.15.102:0/586480175 >> v1:172.21.15.102:6816/34937 conn(0x7fef30031160 legacy=0x7fef30033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:22.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.217+0000 7fef5571d640 1 -- 172.21.15.102:0/586480175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef5009ec20 msgr2=0x7fef50116f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:22.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.217+0000 7fef5571d640 1 --2- 172.21.15.102:0/586480175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef5009ec20 0x7fef50116f40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fef3800dca0 tx=0x7fef3803a000 comp rx=0 tx=0).stop 2024-09-13T17:44:22.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.217+0000 7fef5571d640 1 -- 172.21.15.102:0/586480175 shutdown_connections 2024-09-13T17:44:22.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.217+0000 7fef5571d640 1 --2- 172.21.15.102:0/586480175 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fef501320d0 0x7fef50117480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:22.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.217+0000 7fef5571d640 1 --2- 172.21.15.102:0/586480175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef5009ec20 0x7fef50116f40 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:22.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.217+0000 7fef5571d640 1 --2- 172.21.15.102:0/586480175 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fef5006d8f0 0x7fef5012d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:22.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.217+0000 7fef5571d640 1 -- 172.21.15.102:0/586480175 >> 172.21.15.102:0/586480175 conn(0x7fef50067c00 msgr2=0x7fef50131510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:22.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.217+0000 7fef5571d640 1 -- 172.21.15.102:0/586480175 shutdown_connections 2024-09-13T17:44:22.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.218+0000 7fef5571d640 1 -- 172.21.15.102:0/586480175 wait complete. 2024-09-13T17:44:22.233 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-13T17:44:22.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.333+0000 7f2126700640 1 Processor -- start 2024-09-13T17:44:22.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.333+0000 7f2126700640 1 -- start start 2024-09-13T17:44:22.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.333+0000 7f2126700640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2120096a70 0x7f21200987c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:22.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.333+0000 7f2126700640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2120098d00 0x7f212012c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:22.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.333+0000 7f2126700640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f21200990d0 0x7f212012c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:22.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.333+0000 7f2126700640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f212006a1e0 con 0x7f2120096a70 2024-09-13T17:44:22.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.333+0000 7f2126700640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f212009a900 con 0x7f21200990d0 2024-09-13T17:44:22.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.333+0000 7f2126700640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2120099e80 con 0x7f2120098d00 2024-09-13T17:44:22.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.333+0000 7f211ffff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2120096a70 0x7f21200987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:22.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.333+0000 7f211f7fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2120098d00 0x7f212012c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:22.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.333+0000 7f2124c9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f21200990d0 0x7f212012c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:22.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.333+0000 7f211ffff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f2120096a70 0x7f21200987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55588/0 (socket says 172.21.15.102:55588) 2024-09-13T17:44:22.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.333+0000 7f211f7fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2120098d00 0x7f212012c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:42046/0 (socket says 172.21.15.102:42046) 2024-09-13T17:44:22.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.333+0000 7f211ffff640 1 -- 172.21.15.102:0/3292640401 learned_addr learned my addr 172.21.15.102:0/3292640401 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:22.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.334+0000 7f2124c9d640 1 -- 172.21.15.102:0/3292640401 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2120098d00 msgr2=0x7f212012c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:22.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.334+0000 7f2124c9d640 1 --2- 172.21.15.102:0/3292640401 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2120098d00 0x7f212012c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:22.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.334+0000 7f2124c9d640 1 -- 172.21.15.102:0/3292640401 >> v2:172.21.15.102:3300/0 conn(0x7f2120096a70 msgr2=0x7f21200987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:22.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.334+0000 7f2124c9d640 1 --2- 172.21.15.102:0/3292640401 >> v2:172.21.15.102:3300/0 conn(0x7f2120096a70 0x7f21200987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:22.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.334+0000 7f2124c9d640 1 -- 172.21.15.102:0/3292640401 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f212006eb50 con 0x7f21200990d0 2024-09-13T17:44:22.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.334+0000 7f211ffff640 1 --2- 172.21.15.102:0/3292640401 >> v2:172.21.15.102:3300/0 conn(0x7f2120096a70 0x7f21200987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:22.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.334+0000 7f211f7fe640 1 --2- 172.21.15.102:0/3292640401 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2120098d00 0x7f212012c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:22.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.334+0000 7f2124c9d640 1 --2- 172.21.15.102:0/3292640401 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f21200990d0 0x7f212012c6b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2118009e90 tx=0x7f2118009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28dd93173251f6dd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:22.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.334+0000 7f211effd640 1 -- 172.21.15.102:0/3292640401 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2118031400 con 0x7f21200990d0 2024-09-13T17:44:22.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.334+0000 7f211effd640 1 -- 172.21.15.102:0/3292640401 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f211802d050 con 0x7f21200990d0 2024-09-13T17:44:22.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.334+0000 7f211effd640 1 -- 172.21.15.102:0/3292640401 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21180317d0 con 0x7f21200990d0 2024-09-13T17:44:22.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.335+0000 7f2126700640 1 -- 172.21.15.102:0/3292640401 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f21200990d0 msgr2=0x7f212012c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:22.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.335+0000 7f2126700640 1 --2- 172.21.15.102:0/3292640401 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f21200990d0 0x7f212012c6b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2118009e90 tx=0x7f2118009780 comp rx=0 tx=0).stop 2024-09-13T17:44:22.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.335+0000 7f2126700640 1 -- 172.21.15.102:0/3292640401 shutdown_connections 2024-09-13T17:44:22.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.335+0000 7f2126700640 1 --2- 172.21.15.102:0/3292640401 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f21200990d0 0x7f212012c6b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:22.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.335+0000 7f2126700640 1 --2- 172.21.15.102:0/3292640401 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2120098d00 0x7f212012c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:22.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.335+0000 7f2126700640 1 --2- 172.21.15.102:0/3292640401 >> v2:172.21.15.102:3300/0 conn(0x7f2120096a70 0x7f21200987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:22.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.335+0000 7f2126700640 1 -- 172.21.15.102:0/3292640401 >> 172.21.15.102:0/3292640401 conn(0x7f212009e720 msgr2=0x7f212009eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:22.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.335+0000 7f2126700640 1 -- 172.21.15.102:0/3292640401 shutdown_connections 2024-09-13T17:44:22.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.335+0000 7f2126700640 1 -- 172.21.15.102:0/3292640401 wait complete. 2024-09-13T17:44:22.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.336+0000 7f2126700640 1 Processor -- start 2024-09-13T17:44:22.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.336+0000 7f2126700640 1 -- start start 2024-09-13T17:44:22.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.336+0000 7f2126700640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2120096a70 0x7f21201202b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:22.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.336+0000 7f2126700640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2120098d00 0x7f212012c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:22.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.336+0000 7f2126700640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21200990d0 0x7f212011c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:22.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.336+0000 7f2126700640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212011e940 con 0x7f21200990d0 2024-09-13T17:44:22.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.336+0000 7f2126700640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f212009a050 con 0x7f2120098d00 2024-09-13T17:44:22.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.336+0000 7f2126700640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f212011f810 con 0x7f2120096a70 2024-09-13T17:44:22.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.337+0000 7f211ffff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2120096a70 0x7f21201202b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:22.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.337+0000 7f211f7fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2120098d00 0x7f212012c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:22.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.337+0000 7f211ffff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2120096a70 0x7f21201202b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:42062/0 (socket says 172.21.15.102:42062) 2024-09-13T17:44:22.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.337+0000 7f211ffff640 1 -- 172.21.15.102:0/1055518053 learned_addr learned my addr 172.21.15.102:0/1055518053 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:22.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.337+0000 7f2124c9d640 1 --2- 172.21.15.102:0/1055518053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21200990d0 0x7f212011c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:22.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.337+0000 7f211ffff640 1 -- 172.21.15.102:0/1055518053 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2120098d00 msgr2=0x7f212012c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:22.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.337+0000 7f211ffff640 1 --2- 172.21.15.102:0/1055518053 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2120098d00 0x7f212012c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:22.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.337+0000 7f211ffff640 1 -- 172.21.15.102:0/1055518053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21200990d0 msgr2=0x7f212011c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:22.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.337+0000 7f211ffff640 1 --2- 172.21.15.102:0/1055518053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21200990d0 0x7f212011c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:22.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.337+0000 7f211ffff640 1 -- 172.21.15.102:0/1055518053 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2110009450 con 0x7f2120096a70 2024-09-13T17:44:22.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.337+0000 7f211f7fe640 1 --2- 172.21.15.102:0/1055518053 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2120098d00 0x7f212012c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:22.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.337+0000 7f2124c9d640 1 --2- 172.21.15.102:0/1055518053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21200990d0 0x7f212011c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:22.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.337+0000 7f211ffff640 1 --2- 172.21.15.102:0/1055518053 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2120096a70 0x7f21201202b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f211000ec80 tx=0x7f21100091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:22.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.337+0000 7f2107fff640 1 -- 172.21.15.102:0/1055518053 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211000eec0 con 0x7f2120096a70 2024-09-13T17:44:22.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.337+0000 7f2107fff640 1 -- 172.21.15.102:0/1055518053 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f211000cbc0 con 0x7f2120096a70 2024-09-13T17:44:22.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.337+0000 7f2107fff640 1 -- 172.21.15.102:0/1055518053 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2110013470 con 0x7f2120096a70 2024-09-13T17:44:22.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.337+0000 7f2126700640 1 -- 172.21.15.102:0/1055518053 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2114009390 con 0x7f2120096a70 2024-09-13T17:44:22.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.337+0000 7f2126700640 1 -- 172.21.15.102:0/1055518053 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2118009a20 con 0x7f2120096a70 2024-09-13T17:44:22.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.338+0000 7f2107fff640 1 -- 172.21.15.102:0/1055518053 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f2110013820 con 0x7f2120096a70 2024-09-13T17:44:22.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.338+0000 7f2107fff640 1 -- 172.21.15.102:0/1055518053 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 3265+0+0 (secure 0 0 0) 0x7f21100422e0 con 0x7f2120096a70 2024-09-13T17:44:22.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.338+0000 7f2126700640 1 -- 172.21.15.102:0/1055518053 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2120064000 con 0x7f2120096a70 2024-09-13T17:44:22.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.343+0000 7f2107fff640 1 -- 172.21.15.102:0/1055518053 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f2110021070 con 0x7f2120096a70 2024-09-13T17:44:22.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:22.497+0000 7f2126700640 1 -- 172.21.15.102:0/1055518053 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f212011fa30 con 0x7f2120096a70 2024-09-13T17:44:23.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.219+0000 7f2107fff640 1 -- 172.21.15.102:0/1055518053 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v35) v1 ==== 169+0+0 (secure 0 0 0) 0x7f21100236b0 con 0x7f2120096a70 2024-09-13T17:44:23.222 INFO:teuthology.orchestra.run.smithi102.stderr:pool 'cephfs_data' removed 2024-09-13T17:44:23.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.224+0000 7f2126700640 1 -- 172.21.15.102:0/1055518053 >> v1:172.21.15.102:6816/34937 conn(0x7f210c031160 legacy=0x7f210c033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:23.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.224+0000 7f2126700640 1 -- 172.21.15.102:0/1055518053 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2120096a70 msgr2=0x7f21201202b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:23.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.224+0000 7f2126700640 1 --2- 172.21.15.102:0/1055518053 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2120096a70 0x7f21201202b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f211000ec80 tx=0x7f21100091d0 comp rx=0 tx=0).stop 2024-09-13T17:44:23.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.224+0000 7f2126700640 1 -- 172.21.15.102:0/1055518053 shutdown_connections 2024-09-13T17:44:23.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.224+0000 7f2126700640 1 --2- 172.21.15.102:0/1055518053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21200990d0 0x7f212011c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:23.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.224+0000 7f2126700640 1 --2- 172.21.15.102:0/1055518053 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f2120098d00 0x7f212012c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:23.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.224+0000 7f2126700640 1 --2- 172.21.15.102:0/1055518053 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f2120096a70 0x7f21201202b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:23.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.224+0000 7f2126700640 1 -- 172.21.15.102:0/1055518053 >> 172.21.15.102:0/1055518053 conn(0x7f212009e720 msgr2=0x7f2120097210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:23.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.224+0000 7f2126700640 1 -- 172.21.15.102:0/1055518053 shutdown_connections 2024-09-13T17:44:23.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.224+0000 7f2126700640 1 -- 172.21.15.102:0/1055518053 wait complete. 2024-09-13T17:44:23.242 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-13T17:44:23.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.340+0000 7f81d195d640 1 Processor -- start 2024-09-13T17:44:23.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.340+0000 7f81d195d640 1 -- start start 2024-09-13T17:44:23.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.340+0000 7f81d195d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f81cc06ed60 0x7f81cc097300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:23.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.340+0000 7f81d195d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81cc131530 0x7f81cc133910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:23.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.340+0000 7f81d195d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81cc098830 con 0x7f81cc09e160 2024-09-13T17:44:23.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.340+0000 7f81d195d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81cc06a4e0 con 0x7f81cc06ed60 2024-09-13T17:44:23.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.340+0000 7f81d195d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc09e530 con 0x7f81cc131530 2024-09-13T17:44:23.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.340+0000 7f81c3fff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f81cc06ed60 0x7f81cc097300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:23.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.340+0000 7f81cb7fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81cc131530 0x7f81cc133910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:23.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.341+0000 7f81c3fff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f81cc06ed60 0x7f81cc097300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55604/0 (socket says 172.21.15.102:55604) 2024-09-13T17:44:23.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.340+0000 7f81caffd640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f81cc09e160 0x7f81cc096bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:35840/0 (socket says 172.21.15.102:35840) 2024-09-13T17:44:23.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.341+0000 7f81c3fff640 1 -- 172.21.15.102:0/3018405041 learned_addr learned my addr 172.21.15.102:0/3018405041 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:23.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.341+0000 7f81c3fff640 1 -- 172.21.15.102:0/3018405041 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81cc131530 msgr2=0x7f81cc133910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:23.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.341+0000 7f81c3fff640 1 --2- 172.21.15.102:0/3018405041 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81cc131530 0x7f81cc133910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:23.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.341+0000 7f81c3fff640 1 -- 172.21.15.102:0/3018405041 >> v1:172.21.15.102:6789/0 conn(0x7f81cc09e160 legacy=0x7f81cc096bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:23.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.341+0000 7f81c3fff640 1 -- 172.21.15.102:0/3018405041 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81cc098b00 con 0x7f81cc06ed60 2024-09-13T17:44:23.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.341+0000 7f81cb7fe640 1 --2- 172.21.15.102:0/3018405041 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81cc131530 0x7f81cc133910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:23.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.341+0000 7f81c3fff640 1 --2- 172.21.15.102:0/3018405041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81cc06ed60 0x7f81cc097300 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f81b4009910 tx=0x7f81b40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc50681294829cd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:23.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.342+0000 7f81ca7fc640 1 -- 172.21.15.102:0/3018405041 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b402ad20 con 0x7f81cc06ed60 2024-09-13T17:44:23.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.342+0000 7f81ca7fc640 1 -- 172.21.15.102:0/3018405041 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b402ae80 con 0x7f81cc06ed60 2024-09-13T17:44:23.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.342+0000 7f81ca7fc640 1 -- 172.21.15.102:0/3018405041 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b4032670 con 0x7f81cc06ed60 2024-09-13T17:44:23.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.342+0000 7f81d195d640 1 -- 172.21.15.102:0/3018405041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81cc06ed60 msgr2=0x7f81cc097300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:23.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.342+0000 7f81d195d640 1 --2- 172.21.15.102:0/3018405041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81cc06ed60 0x7f81cc097300 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f81b4009910 tx=0x7f81b40091d0 comp rx=0 tx=0).stop 2024-09-13T17:44:23.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.342+0000 7f81d195d640 1 -- 172.21.15.102:0/3018405041 shutdown_connections 2024-09-13T17:44:23.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.342+0000 7f81d195d640 1 --2- 172.21.15.102:0/3018405041 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81cc131530 0x7f81cc133910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:23.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.342+0000 7f81d195d640 1 --2- 172.21.15.102:0/3018405041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81cc06ed60 0x7f81cc097300 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:23.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.342+0000 7f81d195d640 1 -- 172.21.15.102:0/3018405041 >> 172.21.15.102:0/3018405041 conn(0x7f81cc09e8c0 msgr2=0x7f81cc09ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:23.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.342+0000 7f81d195d640 1 -- 172.21.15.102:0/3018405041 shutdown_connections 2024-09-13T17:44:23.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.342+0000 7f81d195d640 1 -- 172.21.15.102:0/3018405041 wait complete. 2024-09-13T17:44:23.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.343+0000 7f81d195d640 1 Processor -- start 2024-09-13T17:44:23.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.343+0000 7f81d195d640 1 -- start start 2024-09-13T17:44:23.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.343+0000 7f81d195d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f81cc06ed60 0x7f81cc129430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:23.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.343+0000 7f81d195d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81cc09e160 0x7f81cc129970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:23.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.343+0000 7f81d195d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81cc131530 0x7f81cc06a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:23.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.343+0000 7f81d195d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc131050 con 0x7f81cc131530 2024-09-13T17:44:23.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.343+0000 7f81d195d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc0980a0 con 0x7f81cc06ed60 2024-09-13T17:44:23.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.343+0000 7f81d195d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc09bc20 con 0x7f81cc09e160 2024-09-13T17:44:23.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.344+0000 7f81c3fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81cc09e160 0x7f81cc129970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:23.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.344+0000 7f81caffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f81cc06ed60 0x7f81cc129430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:23.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.344+0000 7f81cb7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81cc131530 0x7f81cc06a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:23.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.344+0000 7f81c3fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81cc09e160 0x7f81cc129970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:42074/0 (socket says 172.21.15.102:42074) 2024-09-13T17:44:23.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.344+0000 7f81caffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f81cc06ed60 0x7f81cc129430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48718/0 (socket says 172.21.15.102:48718) 2024-09-13T17:44:23.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.344+0000 7f81cb7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81cc131530 0x7f81cc06a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55610/0 (socket says 172.21.15.102:55610) 2024-09-13T17:44:23.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.344+0000 7f81c3fff640 1 -- 172.21.15.102:0/1625350913 learned_addr learned my addr 172.21.15.102:0/1625350913 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:23.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.344+0000 7f81c3fff640 1 -- 172.21.15.102:0/1625350913 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f81cc06ed60 msgr2=0x7f81cc129430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:23.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.344+0000 7f81c3fff640 1 --2- 172.21.15.102:0/1625350913 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f81cc06ed60 0x7f81cc129430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:23.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.344+0000 7f81c3fff640 1 -- 172.21.15.102:0/1625350913 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81cc131530 msgr2=0x7f81cc06a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:23.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.344+0000 7f81c3fff640 1 --2- 172.21.15.102:0/1625350913 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81cc131530 0x7f81cc06a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:23.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.344+0000 7f81c3fff640 1 -- 172.21.15.102:0/1625350913 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c4009a20 con 0x7f81cc09e160 2024-09-13T17:44:23.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.344+0000 7f81cb7fe640 1 --2- 172.21.15.102:0/1625350913 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81cc131530 0x7f81cc06a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:23.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.344+0000 7f81caffd640 1 --2- 172.21.15.102:0/1625350913 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f81cc06ed60 0x7f81cc129430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:23.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.344+0000 7f81c3fff640 1 --2- 172.21.15.102:0/1625350913 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81cc09e160 0x7f81cc129970 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f81b4009970 tx=0x7f81b4032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:23.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.344+0000 7f81c2ffd640 1 -- 172.21.15.102:0/1625350913 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b403b610 con 0x7f81cc09e160 2024-09-13T17:44:23.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.344+0000 7f81c2ffd640 1 -- 172.21.15.102:0/1625350913 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b403b790 con 0x7f81cc09e160 2024-09-13T17:44:23.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.344+0000 7f81d195d640 1 -- 172.21.15.102:0/1625350913 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81b4009450 con 0x7f81cc09e160 2024-09-13T17:44:23.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.344+0000 7f81c2ffd640 1 -- 172.21.15.102:0/1625350913 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b403bb90 con 0x7f81cc09e160 2024-09-13T17:44:23.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.344+0000 7f81d195d640 1 -- 172.21.15.102:0/1625350913 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81cc06cf90 con 0x7f81cc09e160 2024-09-13T17:44:23.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.345+0000 7f81c2ffd640 1 -- 172.21.15.102:0/1625350913 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f81b4043030 con 0x7f81cc09e160 2024-09-13T17:44:23.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.345+0000 7f81c2ffd640 1 -- 172.21.15.102:0/1625350913 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 2851+0+0 (secure 0 0 0) 0x7f81b4041070 con 0x7f81cc09e160 2024-09-13T17:44:23.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.345+0000 7f81d195d640 1 -- 172.21.15.102:0/1625350913 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81cc064000 con 0x7f81cc09e160 2024-09-13T17:44:23.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.349+0000 7f81c2ffd640 1 -- 172.21.15.102:0/1625350913 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f81b4037080 con 0x7f81cc09e160 2024-09-13T17:44:23.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.503+0000 7f81d195d640 1 -- 172.21.15.102:0/1625350913 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f81cc06cc30 con 0x7f81cc09e160 2024-09-13T17:44:23.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.507+0000 7f81c2ffd640 1 -- 172.21.15.102:0/1625350913 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7f81b404b070 con 0x7f81cc09e160 2024-09-13T17:44:23.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.509+0000 7f81d195d640 1 -- 172.21.15.102:0/1625350913 >> v1:172.21.15.102:6816/34937 conn(0x7f81ac031160 legacy=0x7f81ac033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:23.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.509+0000 7f81d195d640 1 -- 172.21.15.102:0/1625350913 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81cc09e160 msgr2=0x7f81cc129970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:23.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.509+0000 7f81d195d640 1 --2- 172.21.15.102:0/1625350913 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81cc09e160 0x7f81cc129970 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f81b4009970 tx=0x7f81b4032000 comp rx=0 tx=0).stop 2024-09-13T17:44:23.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.509+0000 7f81d195d640 1 -- 172.21.15.102:0/1625350913 shutdown_connections 2024-09-13T17:44:23.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.509+0000 7f81d195d640 1 --2- 172.21.15.102:0/1625350913 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81cc131530 0x7f81cc06a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:23.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.509+0000 7f81d195d640 1 --2- 172.21.15.102:0/1625350913 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f81cc09e160 0x7f81cc129970 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:23.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.509+0000 7f81d195d640 1 --2- 172.21.15.102:0/1625350913 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f81cc06ed60 0x7f81cc129430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:23.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.509+0000 7f81d195d640 1 -- 172.21.15.102:0/1625350913 >> 172.21.15.102:0/1625350913 conn(0x7f81cc09e8c0 msgr2=0x7f81cc09b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:23.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.509+0000 7f81d195d640 1 -- 172.21.15.102:0/1625350913 shutdown_connections 2024-09-13T17:44:23.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.509+0000 7f81d195d640 1 -- 172.21.15.102:0/1625350913 wait complete. 2024-09-13T17:44:23.522 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_newops.TestNewOps.test_newops_getvxattr' 2024-09-13T17:44:23.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.618+0000 7f0afdd1d640 1 Processor -- start 2024-09-13T17:44:23.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.618+0000 7f0afdd1d640 1 -- start start 2024-09-13T17:44:23.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.619+0000 7f0afdd1d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0af8068920 0x7f0af809d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:23.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.619+0000 7f0afdd1d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0af809e020 0x7f0af812fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:23.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.619+0000 7f0afdd1d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0af80695a0 con 0x7f0af809dc50 2024-09-13T17:44:23.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.619+0000 7f0afdd1d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0af8068cf0 con 0x7f0af809e020 2024-09-13T17:44:23.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.619+0000 7f0afdd1d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af809e4a0 con 0x7f0af8068920 2024-09-13T17:44:23.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.619+0000 7f0af7fff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f0af809e020 0x7f0af812fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:23.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.619+0000 7f0af77fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0af8068920 0x7f0af809d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:23.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.619+0000 7f0af6ffd640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f0af809dc50 0x7f0af812f310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:35844/0 (socket says 172.21.15.102:35844) 2024-09-13T17:44:23.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.619+0000 7f0af6ffd640 1 -- 172.21.15.102:0/479479345 learned_addr learned my addr 172.21.15.102:0/479479345 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:44:23.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.619+0000 7f0af7fff640 1 -- 172.21.15.102:0/479479345 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0af8068920 msgr2=0x7f0af809d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:23.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.619+0000 7f0af7fff640 1 --2- 172.21.15.102:0/479479345 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0af8068920 0x7f0af809d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:23.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.619+0000 7f0af7fff640 1 -- 172.21.15.102:0/479479345 >> v1:172.21.15.102:6789/0 conn(0x7f0af809dc50 legacy=0x7f0af812f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:23.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.619+0000 7f0af7fff640 1 -- 172.21.15.102:0/479479345 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af809f090 con 0x7f0af809e020 2024-09-13T17:44:23.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.619+0000 7f0af67fc640 1 -- 172.21.15.102:0/479479345 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 200158736 0 0) 0x7f0af80695a0 con 0x7f0af809dc50 2024-09-13T17:44:23.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.619+0000 7f0af77fe640 1 --2- 172.21.15.102:0/479479345 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0af8068920 0x7f0af809d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:23.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.620+0000 7f0af7fff640 1 --2- 172.21.15.102:0/479479345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0af809e020 0x7f0af812fa20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0af0009850 tx=0x7f0af00090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11682f7bd86a815a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:23.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.620+0000 7f0af67fc640 1 -- 172.21.15.102:0/479479345 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af002ac80 con 0x7f0af809e020 2024-09-13T17:44:23.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.620+0000 7f0af67fc640 1 -- 172.21.15.102:0/479479345 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af002ade0 con 0x7f0af809e020 2024-09-13T17:44:23.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.620+0000 7f0af67fc640 1 -- 172.21.15.102:0/479479345 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af0032670 con 0x7f0af809e020 2024-09-13T17:44:23.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.621+0000 7f0afdd1d640 1 -- 172.21.15.102:0/479479345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0af809e020 msgr2=0x7f0af812fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:23.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.621+0000 7f0afdd1d640 1 --2- 172.21.15.102:0/479479345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0af809e020 0x7f0af812fa20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0af0009850 tx=0x7f0af00090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:23.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.621+0000 7f0afdd1d640 1 -- 172.21.15.102:0/479479345 shutdown_connections 2024-09-13T17:44:23.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.621+0000 7f0afdd1d640 1 --2- 172.21.15.102:0/479479345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0af809e020 0x7f0af812fa20 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:23.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.621+0000 7f0afdd1d640 1 --2- 172.21.15.102:0/479479345 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0af8068920 0x7f0af809d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:23.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.621+0000 7f0afdd1d640 1 -- 172.21.15.102:0/479479345 >> 172.21.15.102:0/479479345 conn(0x7f0af806f320 msgr2=0x7f0af80a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:23.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.621+0000 7f0afdd1d640 1 -- 172.21.15.102:0/479479345 shutdown_connections 2024-09-13T17:44:23.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.621+0000 7f0afdd1d640 1 -- 172.21.15.102:0/479479345 wait complete. 2024-09-13T17:44:23.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.622+0000 7f0afdd1d640 1 Processor -- start 2024-09-13T17:44:23.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.622+0000 7f0afdd1d640 1 -- start start 2024-09-13T17:44:23.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.622+0000 7f0afdd1d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0af8068920 0x7f0af811e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:23.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.622+0000 7f0afdd1d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0af809dc50 0x7f0af811afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:23.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.622+0000 7f0afdd1d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0af809e020 0x7f0af811b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:23.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.622+0000 7f0afdd1d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af811c2e0 con 0x7f0af8068920 2024-09-13T17:44:23.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.622+0000 7f0afdd1d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af80a1cd0 con 0x7f0af809e020 2024-09-13T17:44:23.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.622+0000 7f0afdd1d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0af809e620 con 0x7f0af809dc50 2024-09-13T17:44:23.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.622+0000 7f0af77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0af8068920 0x7f0af811e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:23.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.622+0000 7f0af7fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0af809e020 0x7f0af811b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:23.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.622+0000 7f0af7fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0af809e020 0x7f0af811b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48726/0 (socket says 172.21.15.102:48726) 2024-09-13T17:44:23.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.622+0000 7f0af7fff640 1 -- 172.21.15.102:0/829173475 learned_addr learned my addr 172.21.15.102:0/829173475 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:23.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.622+0000 7f0af77fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0af8068920 0x7f0af811e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55624/0 (socket says 172.21.15.102:55624) 2024-09-13T17:44:23.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.622+0000 7f0af6ffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0af809dc50 0x7f0af811afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:23.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.623+0000 7f0af77fe640 1 -- 172.21.15.102:0/829173475 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0af809dc50 msgr2=0x7f0af811afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:23.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.623+0000 7f0af77fe640 1 --2- 172.21.15.102:0/829173475 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0af809dc50 0x7f0af811afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:23.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.623+0000 7f0af77fe640 1 -- 172.21.15.102:0/829173475 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0af809e020 msgr2=0x7f0af811b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:23.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.623+0000 7f0af77fe640 1 --2- 172.21.15.102:0/829173475 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0af809e020 0x7f0af811b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:23.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.623+0000 7f0af77fe640 1 -- 172.21.15.102:0/829173475 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae8009a20 con 0x7f0af8068920 2024-09-13T17:44:23.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.623+0000 7f0af7fff640 1 --2- 172.21.15.102:0/829173475 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0af809e020 0x7f0af811b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:23.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.623+0000 7f0af77fe640 1 --2- 172.21.15.102:0/829173475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0af8068920 0x7f0af811e6f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8009fa0 tx=0x7f0ae800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:23.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.623+0000 7f0adf7fe640 1 -- 172.21.15.102:0/829173475 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8009d00 con 0x7f0af8068920 2024-09-13T17:44:23.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.623+0000 7f0adf7fe640 1 -- 172.21.15.102:0/829173475 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae800f040 con 0x7f0af8068920 2024-09-13T17:44:23.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.623+0000 7f0adf7fe640 1 -- 172.21.15.102:0/829173475 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae80146b0 con 0x7f0af8068920 2024-09-13T17:44:23.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.623+0000 7f0afdd1d640 1 -- 172.21.15.102:0/829173475 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af0009390 con 0x7f0af8068920 2024-09-13T17:44:23.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.623+0000 7f0afdd1d640 1 -- 172.21.15.102:0/829173475 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af811cb10 con 0x7f0af8068920 2024-09-13T17:44:23.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.624+0000 7f0adf7fe640 1 -- 172.21.15.102:0/829173475 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f0ae8003020 con 0x7f0af8068920 2024-09-13T17:44:23.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.624+0000 7f0adf7fe640 1 -- 172.21.15.102:0/829173475 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 2851+0+0 (secure 0 0 0) 0x7f0ae8042480 con 0x7f0af8068920 2024-09-13T17:44:23.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.624+0000 7f0afdd1d640 1 -- 172.21.15.102:0/829173475 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af8064000 con 0x7f0af8068920 2024-09-13T17:44:23.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.628+0000 7f0adf7fe640 1 -- 172.21.15.102:0/829173475 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f0ae8011aa0 con 0x7f0af8068920 2024-09-13T17:44:23.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:23.843+0000 7f0afdd1d640 1 -- 172.21.15.102:0/829173475 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_newops.TestNewOps.test_newops_getvxattr"]} v 0) v1 -- 0x7f0af809c7d0 con 0x7f0af8068920 2024-09-13T17:44:24.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.221+0000 7f0adf7fe640 1 -- 172.21.15.102:0/829173475 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_newops.TestNewOps.test_newops_getvxattr"]}]=0 v128) v1 ==== 136+0+0 (secure 0 0 0) 0x7f0ae80112c0 con 0x7f0af8068920 2024-09-13T17:44:24.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.223+0000 7f0afdd1d640 1 -- 172.21.15.102:0/829173475 >> v1:172.21.15.102:6816/34937 conn(0x7f0ae4031160 legacy=0x7f0ae4033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:24.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.223+0000 7f0afdd1d640 1 -- 172.21.15.102:0/829173475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0af8068920 msgr2=0x7f0af811e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:24.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.223+0000 7f0afdd1d640 1 --2- 172.21.15.102:0/829173475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0af8068920 0x7f0af811e6f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8009fa0 tx=0x7f0ae800b040 comp rx=0 tx=0).stop 2024-09-13T17:44:24.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.223+0000 7f0afdd1d640 1 -- 172.21.15.102:0/829173475 shutdown_connections 2024-09-13T17:44:24.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.223+0000 7f0afdd1d640 1 --2- 172.21.15.102:0/829173475 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0af809e020 0x7f0af811b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:24.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.223+0000 7f0afdd1d640 1 --2- 172.21.15.102:0/829173475 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0af809dc50 0x7f0af811afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:24.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.223+0000 7f0afdd1d640 1 --2- 172.21.15.102:0/829173475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0af8068920 0x7f0af811e6f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:24.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.223+0000 7f0afdd1d640 1 -- 172.21.15.102:0/829173475 >> 172.21.15.102:0/829173475 conn(0x7f0af806f320 msgr2=0x7f0af8132460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:24.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.223+0000 7f0afdd1d640 1 -- 172.21.15.102:0/829173475 shutdown_connections 2024-09-13T17:44:24.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.223+0000 7f0afdd1d640 1 -- 172.21.15.102:0/829173475 wait complete. 2024-09-13T17:44:24.235 INFO:tasks.cephfs_test_runner:For nautilus it will crash the MDSs when receive unknown OPs, as a workaround ... ok 2024-09-13T17:44:24.235 INFO:tasks.cephfs_test_runner: 2024-09-13T17:44:24.235 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-13T17:44:24.235 INFO:tasks.cephfs_test_runner:Ran 1 test in 36.187s 2024-09-13T17:44:24.236 INFO:tasks.cephfs_test_runner: 2024-09-13T17:44:24.236 INFO:tasks.cephfs_test_runner:OK 2024-09-13T17:44:24.236 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-13T17:44:24.246 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-13T17:44:24.254 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-13T17:44:24.254 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T17:44:24.254 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:44:24.274 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056027,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=396 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-09-13T17:44:24.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:24.274 DEBUG:teuthology.orchestra.run.smithi171:> ip netns list 2024-09-13T17:44:24.332 INFO:teuthology.orchestra.run.smithi171.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T17:44:24.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:24.333 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T17:44:24.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:24.403 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip link delete ceph-brx 2024-09-13T17:44:24.501 DEBUG:teuthology.run_tasks:Unwinding manager install.upgrade 2024-09-13T17:44:24.511 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-13T17:44:24.521 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-13T17:44:24.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:24.521 DEBUG:teuthology.orchestra.run.smithi171:> ip netns list 2024-09-13T17:44:24.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:24.539 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip link delete ceph-brx 2024-09-13T17:44:24.609 INFO:teuthology.orchestra.run.smithi171.stderr:Cannot find device "ceph-brx" 2024-09-13T17:44:24.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:44:24.611 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-13T17:44:24.623 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-13T17:44:24.624 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T17:44:24.624 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:44:24.668 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056027,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=396 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 2024-09-13T17:44:24.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:24.669 DEBUG:teuthology.orchestra.run.smithi171:> ip netns list 2024-09-13T17:44:24.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:24.724 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip link delete ceph-brx 2024-09-13T17:44:24.790 INFO:teuthology.orchestra.run.smithi171.stderr:Cannot find device "ceph-brx" 2024-09-13T17:44:24.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:44:24.792 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-13T17:44:24.804 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T17:44:24.804 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:44:24.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.903+0000 7fa0802f8640 1 Processor -- start 2024-09-13T17:44:24.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.903+0000 7fa0802f8640 1 -- start start 2024-09-13T17:44:24.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.904+0000 7fa0802f8640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa078128330 0x7fa078128700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:24.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.904+0000 7fa0802f8640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa078128c40 0x7fa07812b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:24.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.904+0000 7fa0802f8640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa078097be0 con 0x7fa07812bd80 2024-09-13T17:44:24.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.904+0000 7fa0802f8640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa078098e70 con 0x7fa078128c40 2024-09-13T17:44:24.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.904+0000 7fa0802f8640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa078098310 con 0x7fa078128330 2024-09-13T17:44:24.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.904+0000 7fa07e094640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa078128330 0x7fa078128700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:24.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.904+0000 7fa07d893640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa078128c40 0x7fa07812b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:24.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.904+0000 7fa07e094640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa078128330 0x7fa078128700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48728/0 (socket says 172.21.15.102:48728) 2024-09-13T17:44:24.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.904+0000 7fa07e094640 1 -- 172.21.15.102:0/2649597484 learned_addr learned my addr 172.21.15.102:0/2649597484 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:24.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.904+0000 7fa07d893640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fa078128c40 0x7fa07812b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55632/0 (socket says 172.21.15.102:55632) 2024-09-13T17:44:24.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.904+0000 7fa07e895640 1 --1- 172.21.15.102:0/2649597484 >> v1:172.21.15.102:6789/0 conn(0x7fa07812bd80 0x7fa078132b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:35858/0 (socket says 172.21.15.102:35858) 2024-09-13T17:44:24.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.904+0000 7fa07e094640 1 -- 172.21.15.102:0/2649597484 >> v2:172.21.15.102:3300/0 conn(0x7fa078128c40 msgr2=0x7fa07812b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:24.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.904+0000 7fa07e094640 1 --2- 172.21.15.102:0/2649597484 >> v2:172.21.15.102:3300/0 conn(0x7fa078128c40 0x7fa07812b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:24.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.904+0000 7fa07e094640 1 -- 172.21.15.102:0/2649597484 >> v1:172.21.15.102:6789/0 conn(0x7fa07812bd80 legacy=0x7fa078132b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:24.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.904+0000 7fa07e094640 1 -- 172.21.15.102:0/2649597484 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0780985d0 con 0x7fa078128330 2024-09-13T17:44:24.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.904+0000 7fa07d092640 1 -- 172.21.15.102:0/2649597484 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3112330367 0 0) 0x7fa078097be0 con 0x7fa07812bd80 2024-09-13T17:44:24.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.904+0000 7fa07d893640 1 --2- 172.21.15.102:0/2649597484 >> v2:172.21.15.102:3300/0 conn(0x7fa078128c40 0x7fa07812b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:24.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.905+0000 7fa07e094640 1 --2- 172.21.15.102:0/2649597484 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa078128330 0x7fa078128700 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa07400bbc0 tx=0x7fa07400b460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebd83478cf5c9862 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:24.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.905+0000 7fa07d092640 1 -- 172.21.15.102:0/2649597484 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa074031610 con 0x7fa078128330 2024-09-13T17:44:24.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.905+0000 7fa07d092640 1 -- 172.21.15.102:0/2649597484 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa074031770 con 0x7fa078128330 2024-09-13T17:44:24.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.905+0000 7fa07d092640 1 -- 172.21.15.102:0/2649597484 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa074031b10 con 0x7fa078128330 2024-09-13T17:44:24.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.906+0000 7fa0802f8640 1 -- 172.21.15.102:0/2649597484 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa078128330 msgr2=0x7fa078128700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:24.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.906+0000 7fa0802f8640 1 --2- 172.21.15.102:0/2649597484 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa078128330 0x7fa078128700 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa07400bbc0 tx=0x7fa07400b460 comp rx=0 tx=0).stop 2024-09-13T17:44:24.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.906+0000 7fa0802f8640 1 -- 172.21.15.102:0/2649597484 shutdown_connections 2024-09-13T17:44:24.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.906+0000 7fa0802f8640 1 --2- 172.21.15.102:0/2649597484 >> v2:172.21.15.102:3300/0 conn(0x7fa078128c40 0x7fa07812b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:24.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.906+0000 7fa0802f8640 1 --2- 172.21.15.102:0/2649597484 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa078128330 0x7fa078128700 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:24.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.906+0000 7fa0802f8640 1 -- 172.21.15.102:0/2649597484 >> 172.21.15.102:0/2649597484 conn(0x7fa07809e820 msgr2=0x7fa07809ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:24.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.906+0000 7fa0802f8640 1 -- 172.21.15.102:0/2649597484 shutdown_connections 2024-09-13T17:44:24.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.906+0000 7fa0802f8640 1 -- 172.21.15.102:0/2649597484 wait complete. 2024-09-13T17:44:24.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.907+0000 7fa0802f8640 1 Processor -- start 2024-09-13T17:44:24.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.907+0000 7fa0802f8640 1 -- start start 2024-09-13T17:44:24.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.907+0000 7fa0802f8640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa078128330 0x7fa07813ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:24.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.907+0000 7fa0802f8640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa078128c40 0x7fa07813efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:24.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.907+0000 7fa0802f8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa07812bd80 0x7fa07813f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:24.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.907+0000 7fa0802f8640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07811bb60 con 0x7fa07812bd80 2024-09-13T17:44:24.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.907+0000 7fa0802f8640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa07811ab30 con 0x7fa078128330 2024-09-13T17:44:24.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.907+0000 7fa0802f8640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa07811a470 con 0x7fa078128c40 2024-09-13T17:44:24.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.907+0000 7fa07e094640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa078128330 0x7fa07813ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:24.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.907+0000 7fa07e895640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa07812bd80 0x7fa07813f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:24.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.907+0000 7fa07e094640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa078128330 0x7fa07813ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48736/0 (socket says 172.21.15.102:48736) 2024-09-13T17:44:24.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.907+0000 7fa07e094640 1 -- 172.21.15.102:0/2209580665 learned_addr learned my addr 172.21.15.102:0/2209580665 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:24.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.907+0000 7fa07e895640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa07812bd80 0x7fa07813f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55638/0 (socket says 172.21.15.102:55638) 2024-09-13T17:44:24.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.907+0000 7fa07d893640 1 --2- 172.21.15.102:0/2209580665 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa078128c40 0x7fa07813efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:24.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.907+0000 7fa07e094640 1 -- 172.21.15.102:0/2209580665 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa078128c40 msgr2=0x7fa07813efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:24.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.907+0000 7fa07e094640 1 --2- 172.21.15.102:0/2209580665 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa078128c40 0x7fa07813efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:24.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.907+0000 7fa07e094640 1 -- 172.21.15.102:0/2209580665 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa07812bd80 msgr2=0x7fa07813f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:24.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.907+0000 7fa07e094640 1 --2- 172.21.15.102:0/2209580665 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa07812bd80 0x7fa07813f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:24.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.907+0000 7fa07e094640 1 -- 172.21.15.102:0/2209580665 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa068009390 con 0x7fa078128330 2024-09-13T17:44:24.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.908+0000 7fa07e094640 1 --2- 172.21.15.102:0/2209580665 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa078128330 0x7fa07813ea90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa07400bbc0 tx=0x7fa07403a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:24.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.908+0000 7fa06e7fc640 1 -- 172.21.15.102:0/2209580665 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07403bc70 con 0x7fa078128330 2024-09-13T17:44:24.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.908+0000 7fa06e7fc640 1 -- 172.21.15.102:0/2209580665 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa07403bdd0 con 0x7fa078128330 2024-09-13T17:44:24.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.908+0000 7fa06e7fc640 1 -- 172.21.15.102:0/2209580665 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa074045640 con 0x7fa078128330 2024-09-13T17:44:24.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.908+0000 7fa0802f8640 1 -- 172.21.15.102:0/2209580665 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa07400b700 con 0x7fa078128330 2024-09-13T17:44:24.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.908+0000 7fa0802f8640 1 -- 172.21.15.102:0/2209580665 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa07806ec60 con 0x7fa078128330 2024-09-13T17:44:24.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.909+0000 7fa06e7fc640 1 -- 172.21.15.102:0/2209580665 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fa07404e450 con 0x7fa078128330 2024-09-13T17:44:24.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.909+0000 7fa06e7fc640 1 -- 172.21.15.102:0/2209580665 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 2851+0+0 (secure 0 0 0) 0x7fa074063050 con 0x7fa078128330 2024-09-13T17:44:24.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.909+0000 7fa0802f8640 1 -- 172.21.15.102:0/2209580665 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa07811b980 con 0x7fa078128330 2024-09-13T17:44:24.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:24.913+0000 7fa06e7fc640 1 -- 172.21.15.102:0/2209580665 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fa074039570 con 0x7fa078128330 2024-09-13T17:44:25.064 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.063+0000 7fa0802f8640 1 -- 172.21.15.102:0/2209580665 --> v1:172.21.15.102:6816/34937 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa07811bcd0 con 0x7fa05c031160 2024-09-13T17:44:25.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.064+0000 7fa06e7fc640 1 -- 172.21.15.102:0/2209580665 <== mgr.4101 v1:172.21.15.102:6816/34937 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+39705 (unknown 2965378022 0 2207150084) 0x7fa07811bcd0 con 0x7fa05c031160 2024-09-13T17:44:25.066 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:44:25.067 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-09-13T17:44:25.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.066+0000 7fa0802f8640 1 -- 172.21.15.102:0/2209580665 >> v1:172.21.15.102:6816/34937 conn(0x7fa05c031160 legacy=0x7fa05c033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:25.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.066+0000 7fa0802f8640 1 -- 172.21.15.102:0/2209580665 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa078128330 msgr2=0x7fa07813ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:25.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.066+0000 7fa0802f8640 1 --2- 172.21.15.102:0/2209580665 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa078128330 0x7fa07813ea90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa07400bbc0 tx=0x7fa07403a000 comp rx=0 tx=0).stop 2024-09-13T17:44:25.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.066+0000 7fa0802f8640 1 -- 172.21.15.102:0/2209580665 shutdown_connections 2024-09-13T17:44:25.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.066+0000 7fa0802f8640 1 --2- 172.21.15.102:0/2209580665 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa07812bd80 0x7fa07813f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.066+0000 7fa0802f8640 1 --2- 172.21.15.102:0/2209580665 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fa078128c40 0x7fa07813efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.066+0000 7fa0802f8640 1 --2- 172.21.15.102:0/2209580665 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fa078128330 0x7fa07813ea90 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.066+0000 7fa0802f8640 1 -- 172.21.15.102:0/2209580665 >> 172.21.15.102:0/2209580665 conn(0x7fa07809e820 msgr2=0x7fa078129740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:25.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.066+0000 7fa0802f8640 1 -- 172.21.15.102:0/2209580665 shutdown_connections 2024-09-13T17:44:25.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.066+0000 7fa0802f8640 1 -- 172.21.15.102:0/2209580665 wait complete. 2024-09-13T17:44:25.079 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2024-09-13T17:44:24.442853+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":159,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1191804,"kb_used_data":2372,"kb_used_omap":6,"kb_used_meta":1189369,"kb_avail":376295556,"statfs":{"total":386547056640,"available":385326649344,"internally_reserved":0,"allocated":2428928,"data_stored":1633565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6816,"internal_metadata":1217914208},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2780,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-46,"num_write_kb":-14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000974"},"pg_stats":[{"pgid":"1.0","version":"11'52","reported_seq":98,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T17:44:23.584041+0000","last_change":"2024-09-13T17:41:55.654539+0000","last_active":"2024-09-13T17:44:23.584041+0000","last_peered":"2024-09-13T17:44:23.584041+0000","last_clean":"2024-09-13T17:44:23.584041+0000","last_became_active":"2024-09-13T17:41:55.653536+0000","last_became_peered":"2024-09-13T17:41:55.653536+0000","last_unstale":"2024-09-13T17:44:23.584041+0000","last_undegraded":"2024-09-13T17:44:23.584041+0000","last_fullsized":"2024-09-13T17:44:23.584041+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:54.532965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:54.532965+0000","last_clean_scrub_stamp":"2024-09-13T17:41:54.532965+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:56.662588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":33,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T17:44:20.378161+0000","last_change":"2024-09-13T17:42:02.137356+0000","last_active":"2024-09-13T17:44:20.378161+0000","last_peered":"2024-09-13T17:44:20.378161+0000","last_clean":"2024-09-13T17:44:20.378161+0000","last_became_active":"2024-09-13T17:42:00.116534+0000","last_became_peered":"2024-09-13T17:42:00.116534+0000","last_unstale":"2024-09-13T17:44:20.378161+0000","last_undegraded":"2024-09-13T17:44:20.378161+0000","last_fullsized":"2024-09-13T17:44:20.378161+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:06.665067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000321799,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T17:44:20.377427+0000","last_change":"2024-09-13T17:42:02.137373+0000","last_active":"2024-09-13T17:44:20.377427+0000","last_peered":"2024-09-13T17:44:20.377427+0000","last_clean":"2024-09-13T17:44:20.377427+0000","last_became_active":"2024-09-13T17:42:00.116542+0000","last_became_peered":"2024-09-13T17:42:00.116542+0000","last_unstale":"2024-09-13T17:44:20.377427+0000","last_undegraded":"2024-09-13T17:44:20.377427+0000","last_fullsized":"2024-09-13T17:44:20.377427+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:53.754563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035063,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T17:44:22.216503+0000","last_change":"2024-09-13T17:42:02.615831+0000","last_active":"2024-09-13T17:44:22.216503+0000","last_peered":"2024-09-13T17:44:22.216503+0000","last_clean":"2024-09-13T17:44:22.216503+0000","last_became_active":"2024-09-13T17:42:00.116427+0000","last_became_peered":"2024-09-13T17:42:00.116427+0000","last_unstale":"2024-09-13T17:44:22.216503+0000","last_undegraded":"2024-09-13T17:44:22.216503+0000","last_fullsized":"2024-09-13T17:44:22.216503+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:29:17.848348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000228423,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T17:44:20.377947+0000","last_change":"2024-09-13T17:42:02.138081+0000","last_active":"2024-09-13T17:44:20.377947+0000","last_peered":"2024-09-13T17:44:20.377947+0000","last_clean":"2024-09-13T17:44:20.377947+0000","last_became_active":"2024-09-13T17:42:00.116465+0000","last_became_peered":"2024-09-13T17:42:00.116465+0000","last_unstale":"2024-09-13T17:44:20.377947+0000","last_undegraded":"2024-09-13T17:44:20.377947+0000","last_fullsized":"2024-09-13T17:44:20.377947+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:40:30.342531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001026573,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T17:44:23.583120+0000","last_change":"2024-09-13T17:42:02.135135+0000","last_active":"2024-09-13T17:44:23.583120+0000","last_peered":"2024-09-13T17:44:23.583120+0000","last_clean":"2024-09-13T17:44:23.583120+0000","last_became_active":"2024-09-13T17:42:00.364240+0000","last_became_peered":"2024-09-13T17:42:00.364240+0000","last_unstale":"2024-09-13T17:44:23.583120+0000","last_undegraded":"2024-09-13T17:44:23.583120+0000","last_fullsized":"2024-09-13T17:44:23.583120+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:31:29.382006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575944,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":32,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T17:44:20.378117+0000","last_change":"2024-09-13T17:42:02.138136+0000","last_active":"2024-09-13T17:44:20.378117+0000","last_peered":"2024-09-13T17:44:20.378117+0000","last_clean":"2024-09-13T17:44:20.378117+0000","last_became_active":"2024-09-13T17:42:00.364233+0000","last_became_peered":"2024-09-13T17:42:00.364233+0000","last_unstale":"2024-09-13T17:44:20.378117+0000","last_undegraded":"2024-09-13T17:44:20.378117+0000","last_fullsized":"2024-09-13T17:44:20.378117+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:54.234194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010757130000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":33,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T17:44:23.584175+0000","last_change":"2024-09-13T17:42:02.134963+0000","last_active":"2024-09-13T17:44:23.584175+0000","last_peered":"2024-09-13T17:44:23.584175+0000","last_clean":"2024-09-13T17:44:23.584175+0000","last_became_active":"2024-09-13T17:42:00.364233+0000","last_became_peered":"2024-09-13T17:42:00.364233+0000","last_unstale":"2024-09-13T17:44:23.584175+0000","last_undegraded":"2024-09-13T17:44:23.584175+0000","last_fullsized":"2024-09-13T17:44:23.584175+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:24:31.191067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393223,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":32,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T17:44:20.378537+0000","last_change":"2024-09-13T17:42:02.138362+0000","last_active":"2024-09-13T17:44:20.378537+0000","last_peered":"2024-09-13T17:44:20.378537+0000","last_clean":"2024-09-13T17:44:20.378537+0000","last_became_active":"2024-09-13T17:42:00.364214+0000","last_became_peered":"2024-09-13T17:42:00.364214+0000","last_unstale":"2024-09-13T17:44:20.378537+0000","last_undegraded":"2024-09-13T17:44:20.378537+0000","last_fullsized":"2024-09-13T17:44:20.378537+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:33.976642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012776910000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705697,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":315392,"data_stored":114083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:44:00 2024","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.30399999999999999,"15min":0.30499999999999999},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.39800000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.29299999999999998,"5min":0.317,"15min":0.32100000000000001},"min":{"1min":0.223,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.45300000000000001,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.29799999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:44:00 2024","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.34899999999999998,"15min":0.35299999999999998},"min":{"1min":0.255,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.5,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.214},{"interface":"front","average":{"1min":0.30599999999999999,"5min":0.318,"15min":0.32000000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.55200000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.27600000000000002}]},{"osd":2,"last update":"Fri Sep 13 17:44:00 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.36899999999999999,"15min":0.372},"min":{"1min":0.28000000000000003,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.51100000000000001,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.318},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.33700000000000002,"15min":0.33800000000000002},"min":{"1min":0.23499999999999999,"5min":0.216,"15min":0.216},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.35099999999999998}]}]},{"osd":2,"up_from":9,"seq":38654705697,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":299008,"data_stored":110599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:43:59 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.31,"15min":0.307},"min":{"1min":0.22500000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.34599999999999997,"15min":0.34999999999999998},"min":{"1min":0.22700000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.36499999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:43:59 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35199999999999998,"15min":0.35299999999999998},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.35699999999999998,"15min":0.35799999999999998},"min":{"1min":0.24399999999999999,"5min":0.224,"15min":0.224},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.38100000000000001}]},{"osd":3,"last update":"Fri Sep 13 17:43:59 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.377,"15min":0.379},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.372,"15min":0.371},"min":{"1min":0.28599999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.34100000000000003}]}]},{"osd":1,"up_from":9,"seq":38654705700,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298248,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073592,"statfs":{"total":96636764160,"available":96331358208,"internally_reserved":0,"allocated":909312,"data_stored":704451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:43:56 2024","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.40300000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"last":0.372},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.31,"15min":0.307},"min":{"1min":0.25,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"last":0.35499999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:43:56 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.318,"15min":0.317},"min":{"1min":0.23200000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.251,"5min":0.222,"15min":0.222},"max":{"1min":0.44800000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"last":0.318}]},{"osd":3,"last update":"Fri Sep 13 17:43:56 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.34799999999999998,"15min":0.34999999999999998},"min":{"1min":0.246,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35999999999999999,"15min":0.36099999999999999},"min":{"1min":0.26800000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.46500000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.33500000000000002}]}]},{"osd":0,"up_from":9,"seq":38654705698,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298244,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073596,"statfs":{"total":96636764160,"available":96331362304,"internally_reserved":0,"allocated":905216,"data_stored":704432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:43:57 2024","interfaces":[{"interface":"back","average":{"1min":0.26700000000000002,"5min":0.29999999999999999,"15min":0.30599999999999999},"min":{"1min":0.214,"5min":0.19,"15min":0.19},"max":{"1min":0.38200000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.307,"5min":0.311,"15min":0.311},"min":{"1min":0.22900000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.42299999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"last":0.25900000000000001}]},{"osd":2,"last update":"Fri Sep 13 17:43:57 2024","interfaces":[{"interface":"back","average":{"1min":0.29799999999999999,"5min":0.318,"15min":0.32100000000000001},"min":{"1min":0.23499999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.41599999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.31,"5min":0.32400000000000001,"15min":0.32600000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.437,"5min":0.44600000000000001,"15min":0.44600000000000001},"last":0.30499999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:43:57 2024","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.33200000000000002,"15min":0.33300000000000002},"min":{"1min":0.249,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.39700000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.316,"5min":0.33200000000000002,"15min":0.33500000000000002},"min":{"1min":0.254,"5min":0.248,"15min":0.248},"max":{"1min":0.40999999999999998,"5min":0.504,"15min":0.504},"last":0.29199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:44:25.080 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:44:25.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.172+0000 7f0567eef640 1 Processor -- start 2024-09-13T17:44:25.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.172+0000 7f0567eef640 1 -- start start 2024-09-13T17:44:25.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.172+0000 7f0567eef640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f056009abb0 0x7f056012c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:25.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.172+0000 7f0567eef640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f056012c6a0 0x7f056012cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:25.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.172+0000 7f0567eef640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0560096d80 0x7f0560097220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:25.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.172+0000 7f0567eef640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f056009af80 con 0x7f056012c6a0 2024-09-13T17:44:25.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.172+0000 7f0567eef640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f056012cff0 con 0x7f056009abb0 2024-09-13T17:44:25.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.172+0000 7f0567eef640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f056006a530 con 0x7f0560096d80 2024-09-13T17:44:25.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.172+0000 7f056548a640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f056012c6a0 0x7f056012cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:25.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.172+0000 7f0565c8b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f056009abb0 0x7f056012c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:25.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.172+0000 7f056548a640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f056012c6a0 0x7f056012cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55640/0 (socket says 172.21.15.102:55640) 2024-09-13T17:44:25.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.172+0000 7f0565c8b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f056009abb0 0x7f056012c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48744/0 (socket says 172.21.15.102:48744) 2024-09-13T17:44:25.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.172+0000 7f056548a640 1 -- 172.21.15.102:0/3636523837 learned_addr learned my addr 172.21.15.102:0/3636523837 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:25.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.172+0000 7f056648c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0560096d80 0x7f0560097220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:25.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.173+0000 7f056648c640 1 -- 172.21.15.102:0/3636523837 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f056009abb0 msgr2=0x7f056012c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:25.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.173+0000 7f056648c640 1 --2- 172.21.15.102:0/3636523837 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f056009abb0 0x7f056012c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.173+0000 7f056648c640 1 -- 172.21.15.102:0/3636523837 >> v2:172.21.15.102:3300/0 conn(0x7f056012c6a0 msgr2=0x7f056012cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:25.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.173+0000 7f056648c640 1 --2- 172.21.15.102:0/3636523837 >> v2:172.21.15.102:3300/0 conn(0x7f056012c6a0 0x7f056012cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.173+0000 7f056648c640 1 -- 172.21.15.102:0/3636523837 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0560098be0 con 0x7f0560096d80 2024-09-13T17:44:25.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.173+0000 7f056548a640 1 --2- 172.21.15.102:0/3636523837 >> v2:172.21.15.102:3300/0 conn(0x7f056012c6a0 0x7f056012cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:25.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.173+0000 7f056648c640 1 --2- 172.21.15.102:0/3636523837 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0560096d80 0x7f0560097220 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0558009e90 tx=0x7f0558009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3883cd51c8559ea server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:25.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.173+0000 7f0564c89640 1 -- 172.21.15.102:0/3636523837 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0558007550 con 0x7f0560096d80 2024-09-13T17:44:25.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.174+0000 7f0564c89640 1 -- 172.21.15.102:0/3636523837 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f055802d050 con 0x7f0560096d80 2024-09-13T17:44:25.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.174+0000 7f0564c89640 1 -- 172.21.15.102:0/3636523837 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0558007920 con 0x7f0560096d80 2024-09-13T17:44:25.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.174+0000 7f0567eef640 1 -- 172.21.15.102:0/3636523837 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0560096d80 msgr2=0x7f0560097220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:25.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.174+0000 7f0567eef640 1 --2- 172.21.15.102:0/3636523837 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0560096d80 0x7f0560097220 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0558009e90 tx=0x7f0558009780 comp rx=0 tx=0).stop 2024-09-13T17:44:25.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.174+0000 7f0567eef640 1 -- 172.21.15.102:0/3636523837 shutdown_connections 2024-09-13T17:44:25.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.174+0000 7f0567eef640 1 --2- 172.21.15.102:0/3636523837 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0560096d80 0x7f0560097220 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.174+0000 7f0567eef640 1 --2- 172.21.15.102:0/3636523837 >> v2:172.21.15.102:3300/0 conn(0x7f056012c6a0 0x7f056012cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.174+0000 7f0567eef640 1 --2- 172.21.15.102:0/3636523837 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f056009abb0 0x7f056012c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.174+0000 7f0567eef640 1 -- 172.21.15.102:0/3636523837 >> 172.21.15.102:0/3636523837 conn(0x7f056009e820 msgr2=0x7f056009ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:25.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.174+0000 7f0567eef640 1 -- 172.21.15.102:0/3636523837 shutdown_connections 2024-09-13T17:44:25.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.174+0000 7f0567eef640 1 -- 172.21.15.102:0/3636523837 wait complete. 2024-09-13T17:44:25.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.175+0000 7f0567eef640 1 Processor -- start 2024-09-13T17:44:25.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.175+0000 7f0567eef640 1 -- start start 2024-09-13T17:44:25.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.175+0000 7f0567eef640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0560096d80 0x7f056013e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:25.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.175+0000 7f0567eef640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f056009abb0 0x7f056013eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:25.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.175+0000 7f0567eef640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f056012c6a0 0x7f056013f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:25.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.175+0000 7f0567eef640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056011d750 con 0x7f0560096d80 2024-09-13T17:44:25.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.175+0000 7f0567eef640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f056011ebf0 con 0x7f056009abb0 2024-09-13T17:44:25.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.175+0000 7f0567eef640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f056011ee40 con 0x7f056012c6a0 2024-09-13T17:44:25.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.175+0000 7f0565c8b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0560096d80 0x7f056013e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:25.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.176+0000 7f056548a640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f056009abb0 0x7f056013eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:25.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.176+0000 7f0565c8b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0560096d80 0x7f056013e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55650/0 (socket says 172.21.15.102:55650) 2024-09-13T17:44:25.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.176+0000 7f0565c8b640 1 -- 172.21.15.102:0/1205923641 learned_addr learned my addr 172.21.15.102:0/1205923641 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:25.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.176+0000 7f056648c640 1 --2- 172.21.15.102:0/1205923641 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f056012c6a0 0x7f056013f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:25.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.176+0000 7f056548a640 1 -- 172.21.15.102:0/1205923641 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f056012c6a0 msgr2=0x7f056013f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:25.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.176+0000 7f056548a640 1 --2- 172.21.15.102:0/1205923641 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f056012c6a0 0x7f056013f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.176+0000 7f056548a640 1 -- 172.21.15.102:0/1205923641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0560096d80 msgr2=0x7f056013e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:25.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.176+0000 7f056548a640 1 --2- 172.21.15.102:0/1205923641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0560096d80 0x7f056013e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.176+0000 7f056548a640 1 -- 172.21.15.102:0/1205923641 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f055c00b700 con 0x7f056009abb0 2024-09-13T17:44:25.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.176+0000 7f0565c8b640 1 --2- 172.21.15.102:0/1205923641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0560096d80 0x7f056013e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:25.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.176+0000 7f056548a640 1 --2- 172.21.15.102:0/1205923641 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f056009abb0 0x7f056013eba0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f055c00edc0 tx=0x7f055c00b460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:25.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.176+0000 7f0555ffb640 1 -- 172.21.15.102:0/1205923641 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c00eec0 con 0x7f056009abb0 2024-09-13T17:44:25.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.176+0000 7f0555ffb640 1 -- 172.21.15.102:0/1205923641 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f055c010440 con 0x7f056009abb0 2024-09-13T17:44:25.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.176+0000 7f0567eef640 1 -- 172.21.15.102:0/1205923641 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0558009a20 con 0x7f056009abb0 2024-09-13T17:44:25.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.176+0000 7f0555ffb640 1 -- 172.21.15.102:0/1205923641 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c010800 con 0x7f056009abb0 2024-09-13T17:44:25.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.176+0000 7f0567eef640 1 -- 172.21.15.102:0/1205923641 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f056011f870 con 0x7f056009abb0 2024-09-13T17:44:25.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.177+0000 7f0555ffb640 1 -- 172.21.15.102:0/1205923641 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f055c010bb0 con 0x7f056009abb0 2024-09-13T17:44:25.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.177+0000 7f0555ffb640 1 -- 172.21.15.102:0/1205923641 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 2851+0+0 (secure 0 0 0) 0x7f055c0423d0 con 0x7f056009abb0 2024-09-13T17:44:25.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.177+0000 7f0567eef640 1 -- 172.21.15.102:0/1205923641 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0560098e00 con 0x7f056009abb0 2024-09-13T17:44:25.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.182+0000 7f0555ffb640 1 -- 172.21.15.102:0/1205923641 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f055c021e60 con 0x7f056009abb0 2024-09-13T17:44:25.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.333+0000 7f0567eef640 1 -- 172.21.15.102:0/1205923641 --> v1:172.21.15.102:6816/34937 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f05600981a0 con 0x7f0544031160 2024-09-13T17:44:25.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.335+0000 7f0555ffb640 1 -- 172.21.15.102:0/1205923641 <== mgr.4101 v1:172.21.15.102:6816/34937 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+39705 (unknown 2965378022 0 2207150084) 0x7f05600981a0 con 0x7f0544031160 2024-09-13T17:44:25.337 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:44:25.337 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-09-13T17:44:25.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.337+0000 7f0567eef640 1 -- 172.21.15.102:0/1205923641 >> v1:172.21.15.102:6816/34937 conn(0x7f0544031160 legacy=0x7f0544033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:25.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.337+0000 7f0567eef640 1 -- 172.21.15.102:0/1205923641 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f056009abb0 msgr2=0x7f056013eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:25.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.337+0000 7f0567eef640 1 --2- 172.21.15.102:0/1205923641 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f056009abb0 0x7f056013eba0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f055c00edc0 tx=0x7f055c00b460 comp rx=0 tx=0).stop 2024-09-13T17:44:25.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.337+0000 7f0567eef640 1 -- 172.21.15.102:0/1205923641 shutdown_connections 2024-09-13T17:44:25.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.337+0000 7f0567eef640 1 --2- 172.21.15.102:0/1205923641 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f056012c6a0 0x7f056013f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.337+0000 7f0567eef640 1 --2- 172.21.15.102:0/1205923641 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f056009abb0 0x7f056013eba0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.337+0000 7f0567eef640 1 --2- 172.21.15.102:0/1205923641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0560096d80 0x7f056013e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.337+0000 7f0567eef640 1 -- 172.21.15.102:0/1205923641 >> 172.21.15.102:0/1205923641 conn(0x7f056009e820 msgr2=0x7f056009b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:25.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.337+0000 7f0567eef640 1 -- 172.21.15.102:0/1205923641 shutdown_connections 2024-09-13T17:44:25.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.337+0000 7f0567eef640 1 -- 172.21.15.102:0/1205923641 wait complete. 2024-09-13T17:44:25.349 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2024-09-13T17:44:24.442853+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":159,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1191804,"kb_used_data":2372,"kb_used_omap":6,"kb_used_meta":1189369,"kb_avail":376295556,"statfs":{"total":386547056640,"available":385326649344,"internally_reserved":0,"allocated":2428928,"data_stored":1633565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6816,"internal_metadata":1217914208},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2780,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-46,"num_write_kb":-14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000974"},"pg_stats":[{"pgid":"1.0","version":"11'52","reported_seq":98,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T17:44:23.584041+0000","last_change":"2024-09-13T17:41:55.654539+0000","last_active":"2024-09-13T17:44:23.584041+0000","last_peered":"2024-09-13T17:44:23.584041+0000","last_clean":"2024-09-13T17:44:23.584041+0000","last_became_active":"2024-09-13T17:41:55.653536+0000","last_became_peered":"2024-09-13T17:41:55.653536+0000","last_unstale":"2024-09-13T17:44:23.584041+0000","last_undegraded":"2024-09-13T17:44:23.584041+0000","last_fullsized":"2024-09-13T17:44:23.584041+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:54.532965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:54.532965+0000","last_clean_scrub_stamp":"2024-09-13T17:41:54.532965+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:56.662588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":33,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T17:44:20.378161+0000","last_change":"2024-09-13T17:42:02.137356+0000","last_active":"2024-09-13T17:44:20.378161+0000","last_peered":"2024-09-13T17:44:20.378161+0000","last_clean":"2024-09-13T17:44:20.378161+0000","last_became_active":"2024-09-13T17:42:00.116534+0000","last_became_peered":"2024-09-13T17:42:00.116534+0000","last_unstale":"2024-09-13T17:44:20.378161+0000","last_undegraded":"2024-09-13T17:44:20.378161+0000","last_fullsized":"2024-09-13T17:44:20.378161+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:06.665067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000321799,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T17:44:20.377427+0000","last_change":"2024-09-13T17:42:02.137373+0000","last_active":"2024-09-13T17:44:20.377427+0000","last_peered":"2024-09-13T17:44:20.377427+0000","last_clean":"2024-09-13T17:44:20.377427+0000","last_became_active":"2024-09-13T17:42:00.116542+0000","last_became_peered":"2024-09-13T17:42:00.116542+0000","last_unstale":"2024-09-13T17:44:20.377427+0000","last_undegraded":"2024-09-13T17:44:20.377427+0000","last_fullsized":"2024-09-13T17:44:20.377427+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:53.754563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035063,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T17:44:22.216503+0000","last_change":"2024-09-13T17:42:02.615831+0000","last_active":"2024-09-13T17:44:22.216503+0000","last_peered":"2024-09-13T17:44:22.216503+0000","last_clean":"2024-09-13T17:44:22.216503+0000","last_became_active":"2024-09-13T17:42:00.116427+0000","last_became_peered":"2024-09-13T17:42:00.116427+0000","last_unstale":"2024-09-13T17:44:22.216503+0000","last_undegraded":"2024-09-13T17:44:22.216503+0000","last_fullsized":"2024-09-13T17:44:22.216503+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:29:17.848348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000228423,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T17:44:20.377947+0000","last_change":"2024-09-13T17:42:02.138081+0000","last_active":"2024-09-13T17:44:20.377947+0000","last_peered":"2024-09-13T17:44:20.377947+0000","last_clean":"2024-09-13T17:44:20.377947+0000","last_became_active":"2024-09-13T17:42:00.116465+0000","last_became_peered":"2024-09-13T17:42:00.116465+0000","last_unstale":"2024-09-13T17:44:20.377947+0000","last_undegraded":"2024-09-13T17:44:20.377947+0000","last_fullsized":"2024-09-13T17:44:20.377947+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:40:30.342531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001026573,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T17:44:23.583120+0000","last_change":"2024-09-13T17:42:02.135135+0000","last_active":"2024-09-13T17:44:23.583120+0000","last_peered":"2024-09-13T17:44:23.583120+0000","last_clean":"2024-09-13T17:44:23.583120+0000","last_became_active":"2024-09-13T17:42:00.364240+0000","last_became_peered":"2024-09-13T17:42:00.364240+0000","last_unstale":"2024-09-13T17:44:23.583120+0000","last_undegraded":"2024-09-13T17:44:23.583120+0000","last_fullsized":"2024-09-13T17:44:23.583120+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:31:29.382006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575944,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":32,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T17:44:20.378117+0000","last_change":"2024-09-13T17:42:02.138136+0000","last_active":"2024-09-13T17:44:20.378117+0000","last_peered":"2024-09-13T17:44:20.378117+0000","last_clean":"2024-09-13T17:44:20.378117+0000","last_became_active":"2024-09-13T17:42:00.364233+0000","last_became_peered":"2024-09-13T17:42:00.364233+0000","last_unstale":"2024-09-13T17:44:20.378117+0000","last_undegraded":"2024-09-13T17:44:20.378117+0000","last_fullsized":"2024-09-13T17:44:20.378117+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:54.234194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010757130000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":33,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T17:44:23.584175+0000","last_change":"2024-09-13T17:42:02.134963+0000","last_active":"2024-09-13T17:44:23.584175+0000","last_peered":"2024-09-13T17:44:23.584175+0000","last_clean":"2024-09-13T17:44:23.584175+0000","last_became_active":"2024-09-13T17:42:00.364233+0000","last_became_peered":"2024-09-13T17:42:00.364233+0000","last_unstale":"2024-09-13T17:44:23.584175+0000","last_undegraded":"2024-09-13T17:44:23.584175+0000","last_fullsized":"2024-09-13T17:44:23.584175+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:24:31.191067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393223,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":32,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T17:44:20.378537+0000","last_change":"2024-09-13T17:42:02.138362+0000","last_active":"2024-09-13T17:44:20.378537+0000","last_peered":"2024-09-13T17:44:20.378537+0000","last_clean":"2024-09-13T17:44:20.378537+0000","last_became_active":"2024-09-13T17:42:00.364214+0000","last_became_peered":"2024-09-13T17:42:00.364214+0000","last_unstale":"2024-09-13T17:44:20.378537+0000","last_undegraded":"2024-09-13T17:44:20.378537+0000","last_fullsized":"2024-09-13T17:44:20.378537+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:33.976642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012776910000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705697,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":315392,"data_stored":114083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:44:00 2024","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.30399999999999999,"15min":0.30499999999999999},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.39800000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.29299999999999998,"5min":0.317,"15min":0.32100000000000001},"min":{"1min":0.223,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.45300000000000001,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.29799999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:44:00 2024","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.34899999999999998,"15min":0.35299999999999998},"min":{"1min":0.255,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.5,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.214},{"interface":"front","average":{"1min":0.30599999999999999,"5min":0.318,"15min":0.32000000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.55200000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.27600000000000002}]},{"osd":2,"last update":"Fri Sep 13 17:44:00 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.36899999999999999,"15min":0.372},"min":{"1min":0.28000000000000003,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.51100000000000001,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.318},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.33700000000000002,"15min":0.33800000000000002},"min":{"1min":0.23499999999999999,"5min":0.216,"15min":0.216},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.35099999999999998}]}]},{"osd":2,"up_from":9,"seq":38654705697,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":299008,"data_stored":110599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:43:59 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.31,"15min":0.307},"min":{"1min":0.22500000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.34599999999999997,"15min":0.34999999999999998},"min":{"1min":0.22700000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.36499999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:43:59 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35199999999999998,"15min":0.35299999999999998},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.35699999999999998,"15min":0.35799999999999998},"min":{"1min":0.24399999999999999,"5min":0.224,"15min":0.224},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.38100000000000001}]},{"osd":3,"last update":"Fri Sep 13 17:43:59 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.377,"15min":0.379},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.372,"15min":0.371},"min":{"1min":0.28599999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.34100000000000003}]}]},{"osd":1,"up_from":9,"seq":38654705700,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298248,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073592,"statfs":{"total":96636764160,"available":96331358208,"internally_reserved":0,"allocated":909312,"data_stored":704451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:43:56 2024","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.40300000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"last":0.372},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.31,"15min":0.307},"min":{"1min":0.25,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"last":0.35499999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:43:56 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.318,"15min":0.317},"min":{"1min":0.23200000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.251,"5min":0.222,"15min":0.222},"max":{"1min":0.44800000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"last":0.318}]},{"osd":3,"last update":"Fri Sep 13 17:43:56 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.34799999999999998,"15min":0.34999999999999998},"min":{"1min":0.246,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35999999999999999,"15min":0.36099999999999999},"min":{"1min":0.26800000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.46500000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.33500000000000002}]}]},{"osd":0,"up_from":9,"seq":38654705698,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298244,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073596,"statfs":{"total":96636764160,"available":96331362304,"internally_reserved":0,"allocated":905216,"data_stored":704432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:43:57 2024","interfaces":[{"interface":"back","average":{"1min":0.26700000000000002,"5min":0.29999999999999999,"15min":0.30599999999999999},"min":{"1min":0.214,"5min":0.19,"15min":0.19},"max":{"1min":0.38200000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.307,"5min":0.311,"15min":0.311},"min":{"1min":0.22900000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.42299999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"last":0.25900000000000001}]},{"osd":2,"last update":"Fri Sep 13 17:43:57 2024","interfaces":[{"interface":"back","average":{"1min":0.29799999999999999,"5min":0.318,"15min":0.32100000000000001},"min":{"1min":0.23499999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.41599999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.31,"5min":0.32400000000000001,"15min":0.32600000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.437,"5min":0.44600000000000001,"15min":0.44600000000000001},"last":0.30499999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:43:57 2024","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.33200000000000002,"15min":0.33300000000000002},"min":{"1min":0.249,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.39700000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.316,"5min":0.33200000000000002,"15min":0.33500000000000002},"min":{"1min":0.254,"5min":0.248,"15min":0.248},"max":{"1min":0.40999999999999998,"5min":0.504,"15min":0.504},"last":0.29199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:44:25.350 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T17:44:25.351 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:44:25.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.442+0000 7f32c151d640 1 Processor -- start 2024-09-13T17:44:25.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.442+0000 7f32c151d640 1 -- start start 2024-09-13T17:44:25.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.443+0000 7f32c151d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32bc09f7f0 0x7f32bc132650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:25.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.443+0000 7f32c151d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f32bc132b90 0x7f32bc134f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:25.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.443+0000 7f32c151d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32bc06c910 con 0x7f32bc06d670 2024-09-13T17:44:25.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.443+0000 7f32c151d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32bc09fbc0 con 0x7f32bc132b90 2024-09-13T17:44:25.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.443+0000 7f32c151d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc06c130 con 0x7f32bc09f7f0 2024-09-13T17:44:25.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.443+0000 7f32b27fc640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32bc09f7f0 0x7f32bc132650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:25.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.443+0000 7f32bb7fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f32bc132b90 0x7f32bc134f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:25.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.443+0000 7f32baffd640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f32bc06d670 0x7f32bc06e620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:35870/0 (socket says 172.21.15.102:35870) 2024-09-13T17:44:25.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.443+0000 7f32baffd640 1 -- 172.21.15.102:0/1631217714 learned_addr learned my addr 172.21.15.102:0/1631217714 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:44:25.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.443+0000 7f32b27fc640 1 -- 172.21.15.102:0/1631217714 >> v2:172.21.15.102:3300/0 conn(0x7f32bc132b90 msgr2=0x7f32bc134f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:25.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.443+0000 7f32b27fc640 1 --2- 172.21.15.102:0/1631217714 >> v2:172.21.15.102:3300/0 conn(0x7f32bc132b90 0x7f32bc134f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.443+0000 7f32b27fc640 1 -- 172.21.15.102:0/1631217714 >> v1:172.21.15.102:6789/0 conn(0x7f32bc06d670 legacy=0x7f32bc06e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:25.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.443+0000 7f32b27fc640 1 -- 172.21.15.102:0/1631217714 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32bc0a0310 con 0x7f32bc09f7f0 2024-09-13T17:44:25.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.443+0000 7f32bb7fe640 1 --2- 172.21.15.102:0/1631217714 >> v2:172.21.15.102:3300/0 conn(0x7f32bc132b90 0x7f32bc134f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:25.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.444+0000 7f32b27fc640 1 --2- 172.21.15.102:0/1631217714 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32bc09f7f0 0x7f32bc132650 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f32a4009850 tx=0x7f32a40090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb2e09470a11c6e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:25.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.444+0000 7f32ba7fc640 1 -- 172.21.15.102:0/1631217714 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a402ad00 con 0x7f32bc09f7f0 2024-09-13T17:44:25.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.444+0000 7f32ba7fc640 1 -- 172.21.15.102:0/1631217714 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a402ae60 con 0x7f32bc09f7f0 2024-09-13T17:44:25.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.444+0000 7f32ba7fc640 1 -- 172.21.15.102:0/1631217714 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a4032670 con 0x7f32bc09f7f0 2024-09-13T17:44:25.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.445+0000 7f32c151d640 1 -- 172.21.15.102:0/1631217714 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32bc09f7f0 msgr2=0x7f32bc132650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:25.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.445+0000 7f32c151d640 1 --2- 172.21.15.102:0/1631217714 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32bc09f7f0 0x7f32bc132650 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f32a4009850 tx=0x7f32a40090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.445+0000 7f32c151d640 1 -- 172.21.15.102:0/1631217714 shutdown_connections 2024-09-13T17:44:25.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.445+0000 7f32c151d640 1 --2- 172.21.15.102:0/1631217714 >> v2:172.21.15.102:3300/0 conn(0x7f32bc132b90 0x7f32bc134f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.445+0000 7f32c151d640 1 --2- 172.21.15.102:0/1631217714 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32bc09f7f0 0x7f32bc132650 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.445+0000 7f32c151d640 1 -- 172.21.15.102:0/1631217714 >> 172.21.15.102:0/1631217714 conn(0x7f32bc067bb0 msgr2=0x7f32bc0a08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:25.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.445+0000 7f32c151d640 1 -- 172.21.15.102:0/1631217714 shutdown_connections 2024-09-13T17:44:25.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.445+0000 7f32c151d640 1 -- 172.21.15.102:0/1631217714 wait complete. 2024-09-13T17:44:25.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.446+0000 7f32c151d640 1 Processor -- start 2024-09-13T17:44:25.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.446+0000 7f32c151d640 1 -- start start 2024-09-13T17:44:25.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.446+0000 7f32c151d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32bc06d670 0x7f32bc11c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:25.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.446+0000 7f32c151d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32bc09f7f0 0x7f32bc1182f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:25.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.446+0000 7f32c151d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32bc132b90 0x7f32bc118830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:25.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.446+0000 7f32c151d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc11a310 con 0x7f32bc06d670 2024-09-13T17:44:25.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.446+0000 7f32c151d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc09da40 con 0x7f32bc09f7f0 2024-09-13T17:44:25.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.446+0000 7f32c151d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc11b440 con 0x7f32bc132b90 2024-09-13T17:44:25.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.446+0000 7f32baffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32bc06d670 0x7f32bc11c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:25.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.446+0000 7f32bb7fe640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32bc132b90 0x7f32bc118830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:25.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.446+0000 7f32baffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32bc06d670 0x7f32bc11c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55660/0 (socket says 172.21.15.102:55660) 2024-09-13T17:44:25.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.446+0000 7f32baffd640 1 -- 172.21.15.102:0/145982313 learned_addr learned my addr 172.21.15.102:0/145982313 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:25.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.446+0000 7f32b27fc640 1 --2- 172.21.15.102:0/145982313 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32bc09f7f0 0x7f32bc1182f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:25.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.447+0000 7f32baffd640 1 -- 172.21.15.102:0/145982313 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32bc132b90 msgr2=0x7f32bc118830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:25.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.447+0000 7f32baffd640 1 --2- 172.21.15.102:0/145982313 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32bc132b90 0x7f32bc118830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.447+0000 7f32baffd640 1 -- 172.21.15.102:0/145982313 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32bc09f7f0 msgr2=0x7f32bc1182f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:25.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.447+0000 7f32baffd640 1 --2- 172.21.15.102:0/145982313 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32bc09f7f0 0x7f32bc1182f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.447+0000 7f32baffd640 1 -- 172.21.15.102:0/145982313 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b4009a20 con 0x7f32bc06d670 2024-09-13T17:44:25.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.447+0000 7f32bb7fe640 1 --2- 172.21.15.102:0/145982313 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32bc132b90 0x7f32bc118830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:25.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.447+0000 7f32b27fc640 1 --2- 172.21.15.102:0/145982313 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32bc09f7f0 0x7f32bc1182f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:25.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.447+0000 7f32baffd640 1 --2- 172.21.15.102:0/145982313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32bc06d670 0x7f32bc11c1a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00c560 tx=0x7f32ac007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:25.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.447+0000 7f32b37fe640 1 -- 172.21.15.102:0/145982313 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac005aa0 con 0x7f32bc06d670 2024-09-13T17:44:25.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.447+0000 7f32b37fe640 1 -- 172.21.15.102:0/145982313 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ac005c20 con 0x7f32bc06d670 2024-09-13T17:44:25.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.447+0000 7f32c151d640 1 -- 172.21.15.102:0/145982313 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32a4009390 con 0x7f32bc06d670 2024-09-13T17:44:25.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.447+0000 7f32b37fe640 1 -- 172.21.15.102:0/145982313 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac007320 con 0x7f32bc06d670 2024-09-13T17:44:25.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.447+0000 7f32c151d640 1 -- 172.21.15.102:0/145982313 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32bc118d70 con 0x7f32bc06d670 2024-09-13T17:44:25.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.448+0000 7f32b37fe640 1 -- 172.21.15.102:0/145982313 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f32ac012400 con 0x7f32bc06d670 2024-09-13T17:44:25.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.448+0000 7f32c151d640 1 -- 172.21.15.102:0/145982313 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32bc11bad0 con 0x7f32bc06d670 2024-09-13T17:44:25.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.448+0000 7f32b37fe640 1 -- 172.21.15.102:0/145982313 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 2851+0+0 (secure 0 0 0) 0x7f32ac041040 con 0x7f32bc06d670 2024-09-13T17:44:25.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.452+0000 7f32b37fe640 1 -- 172.21.15.102:0/145982313 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f32ac017070 con 0x7f32bc06d670 2024-09-13T17:44:25.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.600+0000 7f32c151d640 1 -- 172.21.15.102:0/145982313 --> v1:172.21.15.102:6816/34937 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f32bc064600 con 0x7f329c031160 2024-09-13T17:44:25.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.602+0000 7f32b37fe640 1 -- 172.21.15.102:0/145982313 <== mgr.4101 v1:172.21.15.102:6816/34937 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+39705 (unknown 2965378022 0 2207150084) 0x7f32bc064600 con 0x7f329c031160 2024-09-13T17:44:25.604 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:44:25.604 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-09-13T17:44:25.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.603+0000 7f32c151d640 1 -- 172.21.15.102:0/145982313 >> v1:172.21.15.102:6816/34937 conn(0x7f329c031160 legacy=0x7f329c033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:25.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.603+0000 7f32c151d640 1 -- 172.21.15.102:0/145982313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32bc06d670 msgr2=0x7f32bc11c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:25.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.603+0000 7f32c151d640 1 --2- 172.21.15.102:0/145982313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32bc06d670 0x7f32bc11c1a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00c560 tx=0x7f32ac007f40 comp rx=0 tx=0).stop 2024-09-13T17:44:25.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.603+0000 7f32c151d640 1 -- 172.21.15.102:0/145982313 shutdown_connections 2024-09-13T17:44:25.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.603+0000 7f32c151d640 1 --2- 172.21.15.102:0/145982313 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f32bc132b90 0x7f32bc118830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.603+0000 7f32c151d640 1 --2- 172.21.15.102:0/145982313 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f32bc09f7f0 0x7f32bc1182f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.603+0000 7f32c151d640 1 --2- 172.21.15.102:0/145982313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32bc06d670 0x7f32bc11c1a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.603+0000 7f32c151d640 1 -- 172.21.15.102:0/145982313 >> 172.21.15.102:0/145982313 conn(0x7f32bc067bb0 msgr2=0x7f32bc099c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:25.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.604+0000 7f32c151d640 1 -- 172.21.15.102:0/145982313 shutdown_connections 2024-09-13T17:44:25.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.604+0000 7f32c151d640 1 -- 172.21.15.102:0/145982313 wait complete. 2024-09-13T17:44:25.616 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2024-09-13T17:44:24.442853+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":159,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1191804,"kb_used_data":2372,"kb_used_omap":6,"kb_used_meta":1189369,"kb_avail":376295556,"statfs":{"total":386547056640,"available":385326649344,"internally_reserved":0,"allocated":2428928,"data_stored":1633565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6816,"internal_metadata":1217914208},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2780,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-46,"num_write_kb":-14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000974"},"pg_stats":[{"pgid":"1.0","version":"11'52","reported_seq":98,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T17:44:23.584041+0000","last_change":"2024-09-13T17:41:55.654539+0000","last_active":"2024-09-13T17:44:23.584041+0000","last_peered":"2024-09-13T17:44:23.584041+0000","last_clean":"2024-09-13T17:44:23.584041+0000","last_became_active":"2024-09-13T17:41:55.653536+0000","last_became_peered":"2024-09-13T17:41:55.653536+0000","last_unstale":"2024-09-13T17:44:23.584041+0000","last_undegraded":"2024-09-13T17:44:23.584041+0000","last_fullsized":"2024-09-13T17:44:23.584041+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:54.532965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:54.532965+0000","last_clean_scrub_stamp":"2024-09-13T17:41:54.532965+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:56.662588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":33,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T17:44:20.378161+0000","last_change":"2024-09-13T17:42:02.137356+0000","last_active":"2024-09-13T17:44:20.378161+0000","last_peered":"2024-09-13T17:44:20.378161+0000","last_clean":"2024-09-13T17:44:20.378161+0000","last_became_active":"2024-09-13T17:42:00.116534+0000","last_became_peered":"2024-09-13T17:42:00.116534+0000","last_unstale":"2024-09-13T17:44:20.378161+0000","last_undegraded":"2024-09-13T17:44:20.378161+0000","last_fullsized":"2024-09-13T17:44:20.378161+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:06.665067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000321799,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T17:44:20.377427+0000","last_change":"2024-09-13T17:42:02.137373+0000","last_active":"2024-09-13T17:44:20.377427+0000","last_peered":"2024-09-13T17:44:20.377427+0000","last_clean":"2024-09-13T17:44:20.377427+0000","last_became_active":"2024-09-13T17:42:00.116542+0000","last_became_peered":"2024-09-13T17:42:00.116542+0000","last_unstale":"2024-09-13T17:44:20.377427+0000","last_undegraded":"2024-09-13T17:44:20.377427+0000","last_fullsized":"2024-09-13T17:44:20.377427+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:53.754563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035063,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T17:44:22.216503+0000","last_change":"2024-09-13T17:42:02.615831+0000","last_active":"2024-09-13T17:44:22.216503+0000","last_peered":"2024-09-13T17:44:22.216503+0000","last_clean":"2024-09-13T17:44:22.216503+0000","last_became_active":"2024-09-13T17:42:00.116427+0000","last_became_peered":"2024-09-13T17:42:00.116427+0000","last_unstale":"2024-09-13T17:44:22.216503+0000","last_undegraded":"2024-09-13T17:44:22.216503+0000","last_fullsized":"2024-09-13T17:44:22.216503+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:29:17.848348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000228423,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T17:44:20.377947+0000","last_change":"2024-09-13T17:42:02.138081+0000","last_active":"2024-09-13T17:44:20.377947+0000","last_peered":"2024-09-13T17:44:20.377947+0000","last_clean":"2024-09-13T17:44:20.377947+0000","last_became_active":"2024-09-13T17:42:00.116465+0000","last_became_peered":"2024-09-13T17:42:00.116465+0000","last_unstale":"2024-09-13T17:44:20.377947+0000","last_undegraded":"2024-09-13T17:44:20.377947+0000","last_fullsized":"2024-09-13T17:44:20.377947+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:40:30.342531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001026573,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T17:44:23.583120+0000","last_change":"2024-09-13T17:42:02.135135+0000","last_active":"2024-09-13T17:44:23.583120+0000","last_peered":"2024-09-13T17:44:23.583120+0000","last_clean":"2024-09-13T17:44:23.583120+0000","last_became_active":"2024-09-13T17:42:00.364240+0000","last_became_peered":"2024-09-13T17:42:00.364240+0000","last_unstale":"2024-09-13T17:44:23.583120+0000","last_undegraded":"2024-09-13T17:44:23.583120+0000","last_fullsized":"2024-09-13T17:44:23.583120+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:31:29.382006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575944,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":32,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T17:44:20.378117+0000","last_change":"2024-09-13T17:42:02.138136+0000","last_active":"2024-09-13T17:44:20.378117+0000","last_peered":"2024-09-13T17:44:20.378117+0000","last_clean":"2024-09-13T17:44:20.378117+0000","last_became_active":"2024-09-13T17:42:00.364233+0000","last_became_peered":"2024-09-13T17:42:00.364233+0000","last_unstale":"2024-09-13T17:44:20.378117+0000","last_undegraded":"2024-09-13T17:44:20.378117+0000","last_fullsized":"2024-09-13T17:44:20.378117+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:54.234194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010757130000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":33,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T17:44:23.584175+0000","last_change":"2024-09-13T17:42:02.134963+0000","last_active":"2024-09-13T17:44:23.584175+0000","last_peered":"2024-09-13T17:44:23.584175+0000","last_clean":"2024-09-13T17:44:23.584175+0000","last_became_active":"2024-09-13T17:42:00.364233+0000","last_became_peered":"2024-09-13T17:42:00.364233+0000","last_unstale":"2024-09-13T17:44:23.584175+0000","last_undegraded":"2024-09-13T17:44:23.584175+0000","last_fullsized":"2024-09-13T17:44:23.584175+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:24:31.191067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393223,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":32,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T17:44:20.378537+0000","last_change":"2024-09-13T17:42:02.138362+0000","last_active":"2024-09-13T17:44:20.378537+0000","last_peered":"2024-09-13T17:44:20.378537+0000","last_clean":"2024-09-13T17:44:20.378537+0000","last_became_active":"2024-09-13T17:42:00.364214+0000","last_became_peered":"2024-09-13T17:42:00.364214+0000","last_unstale":"2024-09-13T17:44:20.378537+0000","last_undegraded":"2024-09-13T17:44:20.378537+0000","last_fullsized":"2024-09-13T17:44:20.378537+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:33.976642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012776910000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705697,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":315392,"data_stored":114083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:44:00 2024","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.30399999999999999,"15min":0.30499999999999999},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.39800000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.29299999999999998,"5min":0.317,"15min":0.32100000000000001},"min":{"1min":0.223,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.45300000000000001,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.29799999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:44:00 2024","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.34899999999999998,"15min":0.35299999999999998},"min":{"1min":0.255,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.5,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.214},{"interface":"front","average":{"1min":0.30599999999999999,"5min":0.318,"15min":0.32000000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.55200000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.27600000000000002}]},{"osd":2,"last update":"Fri Sep 13 17:44:00 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.36899999999999999,"15min":0.372},"min":{"1min":0.28000000000000003,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.51100000000000001,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.318},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.33700000000000002,"15min":0.33800000000000002},"min":{"1min":0.23499999999999999,"5min":0.216,"15min":0.216},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.35099999999999998}]}]},{"osd":2,"up_from":9,"seq":38654705697,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":299008,"data_stored":110599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:43:59 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.31,"15min":0.307},"min":{"1min":0.22500000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.34599999999999997,"15min":0.34999999999999998},"min":{"1min":0.22700000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.36499999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:43:59 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35199999999999998,"15min":0.35299999999999998},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.35699999999999998,"15min":0.35799999999999998},"min":{"1min":0.24399999999999999,"5min":0.224,"15min":0.224},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.38100000000000001}]},{"osd":3,"last update":"Fri Sep 13 17:43:59 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.377,"15min":0.379},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.372,"15min":0.371},"min":{"1min":0.28599999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.34100000000000003}]}]},{"osd":1,"up_from":9,"seq":38654705700,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298248,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073592,"statfs":{"total":96636764160,"available":96331358208,"internally_reserved":0,"allocated":909312,"data_stored":704451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:43:56 2024","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.40300000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"last":0.372},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.31,"15min":0.307},"min":{"1min":0.25,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"last":0.35499999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:43:56 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.318,"15min":0.317},"min":{"1min":0.23200000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.251,"5min":0.222,"15min":0.222},"max":{"1min":0.44800000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"last":0.318}]},{"osd":3,"last update":"Fri Sep 13 17:43:56 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.34799999999999998,"15min":0.34999999999999998},"min":{"1min":0.246,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35999999999999999,"15min":0.36099999999999999},"min":{"1min":0.26800000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.46500000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.33500000000000002}]}]},{"osd":0,"up_from":9,"seq":38654705698,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298244,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073596,"statfs":{"total":96636764160,"available":96331362304,"internally_reserved":0,"allocated":905216,"data_stored":704432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:43:57 2024","interfaces":[{"interface":"back","average":{"1min":0.26700000000000002,"5min":0.29999999999999999,"15min":0.30599999999999999},"min":{"1min":0.214,"5min":0.19,"15min":0.19},"max":{"1min":0.38200000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.307,"5min":0.311,"15min":0.311},"min":{"1min":0.22900000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.42299999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"last":0.25900000000000001}]},{"osd":2,"last update":"Fri Sep 13 17:43:57 2024","interfaces":[{"interface":"back","average":{"1min":0.29799999999999999,"5min":0.318,"15min":0.32100000000000001},"min":{"1min":0.23499999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.41599999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.31,"5min":0.32400000000000001,"15min":0.32600000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.437,"5min":0.44600000000000001,"15min":0.44600000000000001},"last":0.30499999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:43:57 2024","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.33200000000000002,"15min":0.33300000000000002},"min":{"1min":0.249,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.39700000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.316,"5min":0.33200000000000002,"15min":0.33500000000000002},"min":{"1min":0.254,"5min":0.248,"15min":0.248},"max":{"1min":0.40999999999999998,"5min":0.504,"15min":0.504},"last":0.29199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:44:25.617 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:44:25.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.708+0000 7f0003f1d640 1 Processor -- start 2024-09-13T17:44:25.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.708+0000 7f0003f1d640 1 -- start start 2024-09-13T17:44:25.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.709+0000 7f0003f1d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efffc128330 0x7efffc128700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:25.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.709+0000 7f0003f1d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7efffc128c40 0x7efffc12b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:25.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.709+0000 7f0003f1d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efffc12bd80 0x7efffc12c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:25.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.709+0000 7f0003f1d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efffc12c760 con 0x7efffc128c40 2024-09-13T17:44:25.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.709+0000 7f0003f1d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efffc097be0 con 0x7efffc12bd80 2024-09-13T17:44:25.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.709+0000 7f0003f1d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efffc098e70 con 0x7efffc128330 2024-09-13T17:44:25.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.709+0000 7f00014b8640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7efffc128c40 0x7efffc12b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:25.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.709+0000 7f0001cb9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efffc128330 0x7efffc128700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:25.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.709+0000 7f00024ba640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efffc12bd80 0x7efffc12c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:25.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.709+0000 7f00014b8640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7efffc128c40 0x7efffc12b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55674/0 (socket says 172.21.15.102:55674) 2024-09-13T17:44:25.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.709+0000 7f0001cb9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efffc128330 0x7efffc128700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:42138/0 (socket says 172.21.15.102:42138) 2024-09-13T17:44:25.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.709+0000 7f00014b8640 1 -- 172.21.15.102:0/12993347 learned_addr learned my addr 172.21.15.102:0/12993347 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:25.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.709+0000 7f00024ba640 1 -- 172.21.15.102:0/12993347 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efffc128330 msgr2=0x7efffc128700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:25.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.709+0000 7f00024ba640 1 --2- 172.21.15.102:0/12993347 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efffc128330 0x7efffc128700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.709+0000 7f00024ba640 1 -- 172.21.15.102:0/12993347 >> v2:172.21.15.102:3300/0 conn(0x7efffc128c40 msgr2=0x7efffc12b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:25.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.709+0000 7f00024ba640 1 --2- 172.21.15.102:0/12993347 >> v2:172.21.15.102:3300/0 conn(0x7efffc128c40 0x7efffc12b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.709+0000 7f00024ba640 1 -- 172.21.15.102:0/12993347 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efffc098310 con 0x7efffc12bd80 2024-09-13T17:44:25.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.709+0000 7f00014b8640 1 --2- 172.21.15.102:0/12993347 >> v2:172.21.15.102:3300/0 conn(0x7efffc128c40 0x7efffc12b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:25.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.710+0000 7f00024ba640 1 --2- 172.21.15.102:0/12993347 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efffc12bd80 0x7efffc12c220 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efff4009e60 tx=0x7efff4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=331fef129e924a96 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:25.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.710+0000 7f0000cb7640 1 -- 172.21.15.102:0/12993347 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff402aea0 con 0x7efffc12bd80 2024-09-13T17:44:25.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.710+0000 7f0000cb7640 1 -- 172.21.15.102:0/12993347 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff402d050 con 0x7efffc12bd80 2024-09-13T17:44:25.713 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.710+0000 7f0000cb7640 1 -- 172.21.15.102:0/12993347 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff4007780 con 0x7efffc12bd80 2024-09-13T17:44:25.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.711+0000 7f0003f1d640 1 -- 172.21.15.102:0/12993347 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efffc12bd80 msgr2=0x7efffc12c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:25.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.711+0000 7f0003f1d640 1 --2- 172.21.15.102:0/12993347 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efffc12bd80 0x7efffc12c220 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efff4009e60 tx=0x7efff4009780 comp rx=0 tx=0).stop 2024-09-13T17:44:25.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.711+0000 7f0003f1d640 1 -- 172.21.15.102:0/12993347 shutdown_connections 2024-09-13T17:44:25.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.711+0000 7f0003f1d640 1 --2- 172.21.15.102:0/12993347 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efffc12bd80 0x7efffc12c220 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.711+0000 7f0003f1d640 1 --2- 172.21.15.102:0/12993347 >> v2:172.21.15.102:3300/0 conn(0x7efffc128c40 0x7efffc12b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.711+0000 7f0003f1d640 1 --2- 172.21.15.102:0/12993347 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efffc128330 0x7efffc128700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.711+0000 7f0003f1d640 1 -- 172.21.15.102:0/12993347 >> 172.21.15.102:0/12993347 conn(0x7efffc09e820 msgr2=0x7efffc09ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:25.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.711+0000 7f0003f1d640 1 -- 172.21.15.102:0/12993347 shutdown_connections 2024-09-13T17:44:25.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.711+0000 7f0003f1d640 1 -- 172.21.15.102:0/12993347 wait complete. 2024-09-13T17:44:25.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.711+0000 7f0003f1d640 1 Processor -- start 2024-09-13T17:44:25.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.711+0000 7f0003f1d640 1 -- start start 2024-09-13T17:44:25.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.712+0000 7f0003f1d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efffc128330 0x7efffc114e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:25.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.712+0000 7f0003f1d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efffc128c40 0x7efffc1153c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:25.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.712+0000 7f0003f1d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efffc12bd80 0x7efffc115900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:25.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.712+0000 7f0003f1d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efffc116f30 con 0x7efffc128c40 2024-09-13T17:44:25.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.712+0000 7f0003f1d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efffc09e2f0 con 0x7efffc12bd80 2024-09-13T17:44:25.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.712+0000 7f0003f1d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efffc125f70 con 0x7efffc128330 2024-09-13T17:44:25.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.712+0000 7f00024ba640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efffc12bd80 0x7efffc115900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:25.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.712+0000 7f0001cb9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efffc128330 0x7efffc114e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:25.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.712+0000 7f00014b8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efffc128c40 0x7efffc1153c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:25.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.712+0000 7f0001cb9640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efffc128330 0x7efffc114e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:42154/0 (socket says 172.21.15.102:42154) 2024-09-13T17:44:25.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.712+0000 7f00014b8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efffc128c40 0x7efffc1153c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55684/0 (socket says 172.21.15.102:55684) 2024-09-13T17:44:25.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.712+0000 7f0001cb9640 1 -- 172.21.15.102:0/2986551780 learned_addr learned my addr 172.21.15.102:0/2986551780 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:25.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.712+0000 7f00024ba640 1 --2- 172.21.15.102:0/2986551780 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efffc12bd80 0x7efffc115900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48780/0 (socket says 172.21.15.102:48780) 2024-09-13T17:44:25.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.712+0000 7f0001cb9640 1 -- 172.21.15.102:0/2986551780 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efffc12bd80 msgr2=0x7efffc115900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:25.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.712+0000 7f0001cb9640 1 --2- 172.21.15.102:0/2986551780 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efffc12bd80 0x7efffc115900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.712+0000 7f0001cb9640 1 -- 172.21.15.102:0/2986551780 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efffc128c40 msgr2=0x7efffc1153c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:25.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.712+0000 7f0001cb9640 1 --2- 172.21.15.102:0/2986551780 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efffc128c40 0x7efffc1153c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.712+0000 7f0001cb9640 1 -- 172.21.15.102:0/2986551780 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effec00b700 con 0x7efffc128330 2024-09-13T17:44:25.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.712+0000 7f00024ba640 1 --2- 172.21.15.102:0/2986551780 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efffc12bd80 0x7efffc115900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:25.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.712+0000 7f00014b8640 1 --2- 172.21.15.102:0/2986551780 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efffc128c40 0x7efffc1153c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:25.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.712+0000 7f0001cb9640 1 --2- 172.21.15.102:0/2986551780 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efffc128330 0x7efffc114e80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efff800ea20 tx=0x7efff80090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:25.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.713+0000 7efff1ffb640 1 -- 172.21.15.102:0/2986551780 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff800ca10 con 0x7efffc128330 2024-09-13T17:44:25.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.713+0000 7efff1ffb640 1 -- 172.21.15.102:0/2986551780 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff800cb70 con 0x7efffc128330 2024-09-13T17:44:25.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.713+0000 7f0003f1d640 1 -- 172.21.15.102:0/2986551780 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efff4009a20 con 0x7efffc128330 2024-09-13T17:44:25.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.713+0000 7efff1ffb640 1 -- 172.21.15.102:0/2986551780 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff8013470 con 0x7efffc128330 2024-09-13T17:44:25.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.713+0000 7f0003f1d640 1 -- 172.21.15.102:0/2986551780 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efffc06eaa0 con 0x7efffc128330 2024-09-13T17:44:25.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.714+0000 7efff1ffb640 1 -- 172.21.15.102:0/2986551780 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7efff8013860 con 0x7efffc128330 2024-09-13T17:44:25.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.714+0000 7f0003f1d640 1 -- 172.21.15.102:0/2986551780 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efffc064000 con 0x7efffc128330 2024-09-13T17:44:25.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.714+0000 7efff1ffb640 1 -- 172.21.15.102:0/2986551780 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 2851+0+0 (secure 0 0 0) 0x7efff8042060 con 0x7efffc128330 2024-09-13T17:44:25.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.718+0000 7efff1ffb640 1 -- 172.21.15.102:0/2986551780 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7efff80232f0 con 0x7efffc128330 2024-09-13T17:44:25.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.872+0000 7f0003f1d640 1 -- 172.21.15.102:0/2986551780 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efffc09e0d0 con 0x7efffc128330 2024-09-13T17:44:25.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.872+0000 7efff1ffb640 1 -- 172.21.15.102:0/2986551780 <== mon.2 v2:172.21.15.102:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+8179 (secure 0 0 0) 0x7efff8021070 con 0x7efffc128330 2024-09-13T17:44:25.874 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:44:25.874 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":35,"fsid":"6507a4f0-6c1b-42c0-9c4d-5af24a57491d","created":"2024-09-13T17:41:42.602906+0000","modified":"2024-09-13T17:44:23.212301+0000","last_up_change":"2024-09-13T17:41:53.530626+0000","last_in_change":"2024-09-13T17:41:44.529778+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:54.442612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T17:41:59.076782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"16a9fb3f-c0e8-4252-b5a0-f305e0d6f44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":35327}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":35327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":35327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":35327}]},"public_addr":"172.21.15.102:6800/35327","cluster_addr":"172.21.15.102:6801/35327","heartbeat_back_addr":"172.21.15.102:6803/35327","heartbeat_front_addr":"172.21.15.102:6802/35327","state":["exists","up"]},{"osd":1,"uuid":"d7b7e53f-89ed-4965-adae-314b40049a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":35328}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":35328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":35328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":35328}]},"public_addr":"172.21.15.102:6804/35328","cluster_addr":"172.21.15.102:6805/35328","heartbeat_back_addr":"172.21.15.102:6807/35328","heartbeat_front_addr":"172.21.15.102:6806/35328","state":["exists","up"]},{"osd":2,"uuid":"7c748ced-47e9-4007-aabe-21d4d2a9ed75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":35351}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":35351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":35351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":35351}]},"public_addr":"172.21.15.102:6808/35351","cluster_addr":"172.21.15.102:6809/35351","heartbeat_back_addr":"172.21.15.102:6811/35351","heartbeat_front_addr":"172.21.15.102:6810/35351","state":["exists","up"]},{"osd":3,"uuid":"743a5d12-dd6c-41c6-a461-dd62b868d710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":35352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":35352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":35352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":35352}]},"public_addr":"172.21.15.102:6812/35352","cluster_addr":"172.21.15.102:6813/35352","heartbeat_back_addr":"172.21.15.102:6815/35352","heartbeat_front_addr":"172.21.15.102:6814/35352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.625188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.636044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.680643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:49.741802+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.102:6817/2209249660":"2024-09-14T17:44:20.203753+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:44:25.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.874+0000 7f0003f1d640 1 -- 172.21.15.102:0/2986551780 >> v1:172.21.15.102:6816/34937 conn(0x7effe0031160 legacy=0x7effe0033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:25.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.874+0000 7f0003f1d640 1 -- 172.21.15.102:0/2986551780 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efffc128330 msgr2=0x7efffc114e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:25.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.874+0000 7f0003f1d640 1 --2- 172.21.15.102:0/2986551780 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efffc128330 0x7efffc114e80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efff800ea20 tx=0x7efff80090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.874+0000 7f0003f1d640 1 -- 172.21.15.102:0/2986551780 shutdown_connections 2024-09-13T17:44:25.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.874+0000 7f0003f1d640 1 --2- 172.21.15.102:0/2986551780 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7efffc12bd80 0x7efffc115900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.874+0000 7f0003f1d640 1 --2- 172.21.15.102:0/2986551780 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efffc128c40 0x7efffc1153c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.874+0000 7f0003f1d640 1 --2- 172.21.15.102:0/2986551780 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7efffc128330 0x7efffc114e80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.874+0000 7f0003f1d640 1 -- 172.21.15.102:0/2986551780 >> 172.21.15.102:0/2986551780 conn(0x7efffc09e820 msgr2=0x7efffc129740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:25.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.874+0000 7f0003f1d640 1 -- 172.21.15.102:0/2986551780 shutdown_connections 2024-09-13T17:44:25.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:25.874+0000 7f0003f1d640 1 -- 172.21.15.102:0/2986551780 wait complete. 2024-09-13T17:44:26.888 INFO:tasks.ceph:Scrubbing osd.0 2024-09-13T17:44:26.888 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-13T17:44:26.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.993+0000 7f305364f640 1 Processor -- start 2024-09-13T17:44:26.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.993+0000 7f305364f640 1 -- start start 2024-09-13T17:44:26.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.994+0000 7f305364f640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f304c09a620 0x7f304c096e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:26.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.994+0000 7f305364f640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f304c0973d0 0x7f304c0977a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:26.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.994+0000 7f305364f640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f304c06a7a0 con 0x7f304c12d280 2024-09-13T17:44:26.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.994+0000 7f305364f640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304c09a9f0 con 0x7f304c09a620 2024-09-13T17:44:26.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.994+0000 7f305364f640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f304c0981e0 con 0x7f304c0973d0 2024-09-13T17:44:26.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.994+0000 7f30513eb640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f304c09a620 0x7f304c096e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:26.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.994+0000 7f3050bea640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f304c0973d0 0x7f304c0977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:26.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.994+0000 7f30513eb640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f304c09a620 0x7f304c096e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48796/0 (socket says 172.21.15.102:48796) 2024-09-13T17:44:26.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.994+0000 7f3050bea640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f304c0973d0 0x7f304c0977a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:42160/0 (socket says 172.21.15.102:42160) 2024-09-13T17:44:26.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.994+0000 7f30513eb640 1 -- 172.21.15.102:0/2835668890 learned_addr learned my addr 172.21.15.102:0/2835668890 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:26.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.994+0000 7f3051bec640 1 --1- 172.21.15.102:0/2835668890 >> v1:172.21.15.102:6789/0 conn(0x7f304c12d280 0x7f304c12d700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:35886/0 (socket says 172.21.15.102:35886) 2024-09-13T17:44:26.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.994+0000 7f3050bea640 1 -- 172.21.15.102:0/2835668890 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f304c09a620 msgr2=0x7f304c096e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:26.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.994+0000 7f3050bea640 1 --2- 172.21.15.102:0/2835668890 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f304c09a620 0x7f304c096e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:26.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.994+0000 7f3050bea640 1 -- 172.21.15.102:0/2835668890 >> v1:172.21.15.102:6789/0 conn(0x7f304c12d280 legacy=0x7f304c12d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:26.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.994+0000 7f3050bea640 1 -- 172.21.15.102:0/2835668890 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304c096b80 con 0x7f304c0973d0 2024-09-13T17:44:26.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.994+0000 7f3043fff640 1 -- 172.21.15.102:0/2835668890 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 330749469 0 0) 0x7f304c06a7a0 con 0x7f304c12d280 2024-09-13T17:44:26.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.994+0000 7f30513eb640 1 --2- 172.21.15.102:0/2835668890 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f304c09a620 0x7f304c096e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:27.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.995+0000 7f3050bea640 1 --2- 172.21.15.102:0/2835668890 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f304c0973d0 0x7f304c0977a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f303c0098b0 tx=0x7f303c0090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d62e254586872c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:27.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.995+0000 7f3043fff640 1 -- 172.21.15.102:0/2835668890 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c02ad50 con 0x7f304c0973d0 2024-09-13T17:44:27.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.995+0000 7f3043fff640 1 -- 172.21.15.102:0/2835668890 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f303c032440 con 0x7f304c0973d0 2024-09-13T17:44:27.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.995+0000 7f3043fff640 1 -- 172.21.15.102:0/2835668890 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c0327e0 con 0x7f304c0973d0 2024-09-13T17:44:27.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.996+0000 7f305364f640 1 -- 172.21.15.102:0/2835668890 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f304c0973d0 msgr2=0x7f304c0977a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.996+0000 7f305364f640 1 --2- 172.21.15.102:0/2835668890 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f304c0973d0 0x7f304c0977a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f303c0098b0 tx=0x7f303c0090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.996+0000 7f305364f640 1 -- 172.21.15.102:0/2835668890 shutdown_connections 2024-09-13T17:44:27.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.996+0000 7f305364f640 1 --2- 172.21.15.102:0/2835668890 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f304c0973d0 0x7f304c0977a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.996+0000 7f305364f640 1 --2- 172.21.15.102:0/2835668890 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f304c09a620 0x7f304c096e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.996+0000 7f305364f640 1 -- 172.21.15.102:0/2835668890 >> 172.21.15.102:0/2835668890 conn(0x7f304c09e720 msgr2=0x7f304c09eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:27.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.996+0000 7f305364f640 1 -- 172.21.15.102:0/2835668890 shutdown_connections 2024-09-13T17:44:27.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.996+0000 7f305364f640 1 -- 172.21.15.102:0/2835668890 wait complete. 2024-09-13T17:44:27.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.997+0000 7f305364f640 1 Processor -- start 2024-09-13T17:44:27.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.997+0000 7f305364f640 1 -- start start 2024-09-13T17:44:27.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.997+0000 7f305364f640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f304c0973d0 0x7f304c11a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.997+0000 7f305364f640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f304c09a620 0x7f304c116a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.997+0000 7f305364f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f304c12d280 0x7f304c116fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.997+0000 7f305364f640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c118060 con 0x7f304c12d280 2024-09-13T17:44:27.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.997+0000 7f305364f640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304c1195a0 con 0x7f304c0973d0 2024-09-13T17:44:27.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.997+0000 7f305364f640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f304c09b8e0 con 0x7f304c09a620 2024-09-13T17:44:27.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.997+0000 7f30513eb640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f304c0973d0 0x7f304c11a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.997+0000 7f30513eb640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f304c0973d0 0x7f304c11a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48798/0 (socket says 172.21.15.102:48798) 2024-09-13T17:44:27.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.997+0000 7f30513eb640 1 -- 172.21.15.102:0/4150661558 learned_addr learned my addr 172.21.15.102:0/4150661558 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:27.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.997+0000 7f3051bec640 1 --2- 172.21.15.102:0/4150661558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f304c12d280 0x7f304c116fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.997+0000 7f3050bea640 1 --2- 172.21.15.102:0/4150661558 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f304c09a620 0x7f304c116a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.998+0000 7f30513eb640 1 -- 172.21.15.102:0/4150661558 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f304c09a620 msgr2=0x7f304c116a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.998+0000 7f30513eb640 1 --2- 172.21.15.102:0/4150661558 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f304c09a620 0x7f304c116a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.998+0000 7f30513eb640 1 -- 172.21.15.102:0/4150661558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f304c12d280 msgr2=0x7f304c116fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.998+0000 7f30513eb640 1 --2- 172.21.15.102:0/4150661558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f304c12d280 0x7f304c116fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.998+0000 7f30513eb640 1 -- 172.21.15.102:0/4150661558 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3048009450 con 0x7f304c0973d0 2024-09-13T17:44:27.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.998+0000 7f3050bea640 1 --2- 172.21.15.102:0/4150661558 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f304c09a620 0x7f304c116a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:27.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.998+0000 7f30513eb640 1 --2- 172.21.15.102:0/4150661558 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f304c0973d0 0x7f304c11a2f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f304800ec80 tx=0x7f30480091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:27.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.998+0000 7f30417fa640 1 -- 172.21.15.102:0/4150661558 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3048009730 con 0x7f304c0973d0 2024-09-13T17:44:27.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.998+0000 7f30417fa640 1 -- 172.21.15.102:0/4150661558 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f304800eec0 con 0x7f304c0973d0 2024-09-13T17:44:27.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.998+0000 7f30417fa640 1 -- 172.21.15.102:0/4150661558 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304800cdf0 con 0x7f304c0973d0 2024-09-13T17:44:27.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.998+0000 7f305364f640 1 -- 172.21.15.102:0/4150661558 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f303c009390 con 0x7f304c0973d0 2024-09-13T17:44:27.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.998+0000 7f305364f640 1 -- 172.21.15.102:0/4150661558 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f304c117510 con 0x7f304c0973d0 2024-09-13T17:44:27.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.999+0000 7f30417fa640 1 -- 172.21.15.102:0/4150661558 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f3048013680 con 0x7f304c0973d0 2024-09-13T17:44:27.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.999+0000 7f305364f640 1 -- 172.21.15.102:0/4150661558 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f304c064000 con 0x7f304c0973d0 2024-09-13T17:44:27.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.999+0000 7f30417fa640 1 -- 172.21.15.102:0/4150661558 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 2851+0+0 (secure 0 0 0) 0x7f3048042490 con 0x7f304c0973d0 2024-09-13T17:44:27.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.999+0000 7f30417fa640 1 -- 172.21.15.102:0/4150661558 --> v1:172.21.15.102:6800/35327 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f303003b440 con 0x7f30300377a0 2024-09-13T17:44:27.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:26.999+0000 7f30417fa640 1 -- 172.21.15.102:0/4150661558 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3048044ac0 con 0x7f304c0973d0 2024-09-13T17:44:27.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.000+0000 7f30417fa640 1 -- 172.21.15.102:0/4150661558 <== osd.0 v1:172.21.15.102:6800/35327 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f303003b440 con 0x7f30300377a0 2024-09-13T17:44:27.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.011+0000 7f305364f640 1 -- 172.21.15.102:0/4150661558 --> v1:172.21.15.102:6800/35327 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f304c06a4e0 con 0x7f30300377a0 2024-09-13T17:44:27.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.012+0000 7f30417fa640 1 -- 172.21.15.102:0/4150661558 <== osd.0 v1:172.21.15.102:6800/35327 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (unknown 0 0 119112422) 0x7f304c06a4e0 con 0x7f30300377a0 2024-09-13T17:44:27.014 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-13T17:44:27.014 INFO:teuthology.orchestra.run.smithi102.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T17:44:27.014 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-13T17:44:27.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.012+0000 7f305364f640 1 -- 172.21.15.102:0/4150661558 >> v1:172.21.15.102:6800/35327 conn(0x7f30300377a0 legacy=0x7f3030039bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.012+0000 7f305364f640 1 -- 172.21.15.102:0/4150661558 >> v1:172.21.15.102:6816/34937 conn(0x7f3030031160 legacy=0x7f3030033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.012+0000 7f305364f640 1 -- 172.21.15.102:0/4150661558 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f304c0973d0 msgr2=0x7f304c11a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.012+0000 7f305364f640 1 --2- 172.21.15.102:0/4150661558 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f304c0973d0 0x7f304c11a2f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f304800ec80 tx=0x7f30480091d0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.012+0000 7f3051bec640 1 -- 172.21.15.102:0/4150661558 reap_dead start 2024-09-13T17:44:27.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.012+0000 7f305364f640 1 -- 172.21.15.102:0/4150661558 shutdown_connections 2024-09-13T17:44:27.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.012+0000 7f305364f640 1 -- 172.21.15.102:0/4150661558 >> 172.21.15.102:0/4150661558 conn(0x7f304c09e720 msgr2=0x7f304c06d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:27.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.012+0000 7f305364f640 1 -- 172.21.15.102:0/4150661558 shutdown_connections 2024-09-13T17:44:27.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.012+0000 7f305364f640 1 -- 172.21.15.102:0/4150661558 wait complete. 2024-09-13T17:44:27.024 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-13T17:44:27.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.117+0000 7feb7ae78640 1 Processor -- start 2024-09-13T17:44:27.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.117+0000 7feb7ae78640 1 -- start start 2024-09-13T17:44:27.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.117+0000 7feb7ae78640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feb7406b140 0x7feb740998c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.117+0000 7feb7ae78640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feb7409ac20 0x7feb74099e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.117+0000 7feb7ae78640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb7406ee90 con 0x7feb7409a340 2024-09-13T17:44:27.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.117+0000 7feb7ae78640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb7406a680 con 0x7feb7409ac20 2024-09-13T17:44:27.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.117+0000 7feb7ae78640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feb7406ac90 con 0x7feb7406b140 2024-09-13T17:44:27.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.118+0000 7feb78c14640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feb7406b140 0x7feb740998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.118+0000 7feb73fff640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feb7409ac20 0x7feb74099e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.118+0000 7feb79415640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7feb7409a340 0x7feb7412d280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:35892/0 (socket says 172.21.15.102:35892) 2024-09-13T17:44:27.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.118+0000 7feb79415640 1 -- 172.21.15.102:0/896353280 learned_addr learned my addr 172.21.15.102:0/896353280 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:44:27.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.118+0000 7feb737fe640 1 -- 172.21.15.102:0/896353280 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2674954908 0 0) 0x7feb7406ee90 con 0x7feb7409a340 2024-09-13T17:44:27.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.118+0000 7feb78c14640 1 -- 172.21.15.102:0/896353280 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feb7409ac20 msgr2=0x7feb74099e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:27.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.118+0000 7feb78c14640 1 --2- 172.21.15.102:0/896353280 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feb7409ac20 0x7feb74099e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.118+0000 7feb78c14640 1 -- 172.21.15.102:0/896353280 >> v1:172.21.15.102:6789/0 conn(0x7feb7409a340 legacy=0x7feb7412d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.118+0000 7feb78c14640 1 -- 172.21.15.102:0/896353280 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb74099510 con 0x7feb7406b140 2024-09-13T17:44:27.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.118+0000 7feb73fff640 1 --2- 172.21.15.102:0/896353280 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feb7409ac20 0x7feb74099e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:27.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.119+0000 7feb78c14640 1 --2- 172.21.15.102:0/896353280 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feb7406b140 0x7feb740998c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7feb68009880 tx=0x7feb680090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2549b99501478d9e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:27.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.119+0000 7feb737fe640 1 -- 172.21.15.102:0/896353280 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6802ac30 con 0x7feb7406b140 2024-09-13T17:44:27.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.119+0000 7feb737fe640 1 -- 172.21.15.102:0/896353280 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb6802ad90 con 0x7feb7406b140 2024-09-13T17:44:27.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.119+0000 7feb737fe640 1 -- 172.21.15.102:0/896353280 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb68032670 con 0x7feb7406b140 2024-09-13T17:44:27.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.119+0000 7feb7ae78640 1 -- 172.21.15.102:0/896353280 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feb7406b140 msgr2=0x7feb740998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.119+0000 7feb7ae78640 1 --2- 172.21.15.102:0/896353280 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feb7406b140 0x7feb740998c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7feb68009880 tx=0x7feb680090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.119+0000 7feb7ae78640 1 -- 172.21.15.102:0/896353280 shutdown_connections 2024-09-13T17:44:27.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.119+0000 7feb7ae78640 1 --2- 172.21.15.102:0/896353280 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feb7409ac20 0x7feb74099e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.119+0000 7feb7ae78640 1 --2- 172.21.15.102:0/896353280 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feb7406b140 0x7feb740998c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.119+0000 7feb7ae78640 1 -- 172.21.15.102:0/896353280 >> 172.21.15.102:0/896353280 conn(0x7feb7409e820 msgr2=0x7feb7409ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:27.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.119+0000 7feb7ae78640 1 -- 172.21.15.102:0/896353280 shutdown_connections 2024-09-13T17:44:27.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.119+0000 7feb7ae78640 1 -- 172.21.15.102:0/896353280 wait complete. 2024-09-13T17:44:27.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.120+0000 7feb7ae78640 1 Processor -- start 2024-09-13T17:44:27.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.120+0000 7feb7ae78640 1 -- start start 2024-09-13T17:44:27.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.121+0000 7feb7ae78640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feb7406b140 0x7feb7411bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.121+0000 7feb7ae78640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feb7409a340 0x7feb7411c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.121+0000 7feb7ae78640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb7409ac20 0x7feb7413c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.121+0000 7feb7ae78640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb74069dd0 con 0x7feb7409ac20 2024-09-13T17:44:27.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.121+0000 7feb7ae78640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb7411e980 con 0x7feb7406b140 2024-09-13T17:44:27.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.121+0000 7feb7ae78640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feb7411ba20 con 0x7feb7409a340 2024-09-13T17:44:27.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.121+0000 7feb73fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feb7409a340 0x7feb7411c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.121+0000 7feb78c14640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feb7406b140 0x7feb7411bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.121+0000 7feb79415640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb7409ac20 0x7feb7413c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.121+0000 7feb73fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feb7409a340 0x7feb7411c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:42186/0 (socket says 172.21.15.102:42186) 2024-09-13T17:44:27.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.121+0000 7feb78c14640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feb7406b140 0x7feb7411bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48810/0 (socket says 172.21.15.102:48810) 2024-09-13T17:44:27.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.121+0000 7feb73fff640 1 -- 172.21.15.102:0/1290412197 learned_addr learned my addr 172.21.15.102:0/1290412197 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:27.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.121+0000 7feb73fff640 1 -- 172.21.15.102:0/1290412197 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feb7406b140 msgr2=0x7feb7411bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.121+0000 7feb73fff640 1 --2- 172.21.15.102:0/1290412197 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feb7406b140 0x7feb7411bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.121+0000 7feb73fff640 1 -- 172.21.15.102:0/1290412197 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb7409ac20 msgr2=0x7feb7413c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.121+0000 7feb73fff640 1 --2- 172.21.15.102:0/1290412197 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb7409ac20 0x7feb7413c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.121+0000 7feb73fff640 1 -- 172.21.15.102:0/1290412197 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb60009390 con 0x7feb7409a340 2024-09-13T17:44:27.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.121+0000 7feb78c14640 1 --2- 172.21.15.102:0/1290412197 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feb7406b140 0x7feb7411bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:27.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.121+0000 7feb73fff640 1 --2- 172.21.15.102:0/1290412197 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feb7409a340 0x7feb7411c200 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7feb6000ec50 tx=0x7feb600090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:27.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.121+0000 7feb70ff9640 1 -- 172.21.15.102:0/1290412197 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6000ee90 con 0x7feb7409a340 2024-09-13T17:44:27.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.121+0000 7feb70ff9640 1 -- 172.21.15.102:0/1290412197 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb6000cbc0 con 0x7feb7409a340 2024-09-13T17:44:27.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.121+0000 7feb7ae78640 1 -- 172.21.15.102:0/1290412197 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb68009390 con 0x7feb7409a340 2024-09-13T17:44:27.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.121+0000 7feb70ff9640 1 -- 172.21.15.102:0/1290412197 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb60013470 con 0x7feb7409a340 2024-09-13T17:44:27.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.121+0000 7feb7ae78640 1 -- 172.21.15.102:0/1290412197 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb7411f670 con 0x7feb7409a340 2024-09-13T17:44:27.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.122+0000 7feb70ff9640 1 -- 172.21.15.102:0/1290412197 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7feb600137e0 con 0x7feb7409a340 2024-09-13T17:44:27.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.122+0000 7feb70ff9640 1 -- 172.21.15.102:0/1290412197 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 2851+0+0 (secure 0 0 0) 0x7feb60042120 con 0x7feb7409a340 2024-09-13T17:44:27.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.122+0000 7feb7ae78640 1 -- 172.21.15.102:0/1290412197 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb74064000 con 0x7feb7409a340 2024-09-13T17:44:27.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.127+0000 7feb70ff9640 1 -- 172.21.15.102:0/1290412197 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7feb60021020 con 0x7feb7409a340 2024-09-13T17:44:27.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.280+0000 7feb7ae78640 1 -- 172.21.15.102:0/1290412197 --> v1:172.21.15.102:6816/34937 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7feb74067690 con 0x7feb58031160 2024-09-13T17:44:27.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.280+0000 7feb70ff9640 1 -- 172.21.15.102:0/1290412197 <== mgr.4101 v1:172.21.15.102:6816/34937 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (unknown 2057319476 0 0) 0x7feb74067690 con 0x7feb58031160 2024-09-13T17:44:27.282 INFO:teuthology.orchestra.run.smithi102.stderr:instructed osd(s) 0 to deep-scrub 2024-09-13T17:44:27.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.282+0000 7feb7ae78640 1 -- 172.21.15.102:0/1290412197 >> v1:172.21.15.102:6816/34937 conn(0x7feb58031160 legacy=0x7feb58033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.282+0000 7feb7ae78640 1 -- 172.21.15.102:0/1290412197 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feb7409a340 msgr2=0x7feb7411c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.282+0000 7feb7ae78640 1 --2- 172.21.15.102:0/1290412197 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feb7409a340 0x7feb7411c200 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7feb6000ec50 tx=0x7feb600090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.282+0000 7feb7ae78640 1 -- 172.21.15.102:0/1290412197 shutdown_connections 2024-09-13T17:44:27.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.282+0000 7feb7ae78640 1 --2- 172.21.15.102:0/1290412197 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb7409ac20 0x7feb7413c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.282+0000 7feb7ae78640 1 --2- 172.21.15.102:0/1290412197 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7feb7409a340 0x7feb7411c200 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.282+0000 7feb7ae78640 1 --2- 172.21.15.102:0/1290412197 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7feb7406b140 0x7feb7411bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.282+0000 7feb7ae78640 1 -- 172.21.15.102:0/1290412197 >> 172.21.15.102:0/1290412197 conn(0x7feb7409e820 msgr2=0x7feb7406cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:27.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.282+0000 7feb7ae78640 1 -- 172.21.15.102:0/1290412197 shutdown_connections 2024-09-13T17:44:27.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.282+0000 7feb7ae78640 1 -- 172.21.15.102:0/1290412197 wait complete. 2024-09-13T17:44:27.294 INFO:tasks.ceph:Scrubbing osd.1 2024-09-13T17:44:27.295 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-13T17:44:27.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.390+0000 7f05756a8640 1 Processor -- start 2024-09-13T17:44:27.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.390+0000 7f05756a8640 1 -- start start 2024-09-13T17:44:27.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.390+0000 7f05756a8640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0570097130 0x7f0570068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.390+0000 7f05756a8640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f05700690e0 0x7f05700694d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.390+0000 7f05756a8640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f057012f440 0x7f057012f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.390+0000 7f05756a8640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0570097500 con 0x7f057012f440 2024-09-13T17:44:27.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.390+0000 7f05756a8640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f057009cf80 con 0x7f0570097130 2024-09-13T17:44:27.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.390+0000 7f05756a8640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0570096c10 con 0x7f05700690e0 2024-09-13T17:44:27.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.391+0000 7f056e7fc640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f05700690e0 0x7f05700694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.391+0000 7f056effd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0570097130 0x7f0570068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.391+0000 7f056f7fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f057012f440 0x7f057012f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.391+0000 7f056effd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0570097130 0x7f0570068ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48824/0 (socket says 172.21.15.102:48824) 2024-09-13T17:44:27.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.391+0000 7f056e7fc640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f05700690e0 0x7f05700694d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:42190/0 (socket says 172.21.15.102:42190) 2024-09-13T17:44:27.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.391+0000 7f056effd640 1 -- 172.21.15.102:0/2277931950 learned_addr learned my addr 172.21.15.102:0/2277931950 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:27.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.391+0000 7f056f7fe640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f057012f440 0x7f057012f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55720/0 (socket says 172.21.15.102:55720) 2024-09-13T17:44:27.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.391+0000 7f056effd640 1 -- 172.21.15.102:0/2277931950 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f05700690e0 msgr2=0x7f05700694d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:27.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.391+0000 7f056effd640 1 --2- 172.21.15.102:0/2277931950 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f05700690e0 0x7f05700694d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.391+0000 7f056effd640 1 -- 172.21.15.102:0/2277931950 >> v2:172.21.15.102:3300/0 conn(0x7f057012f440 msgr2=0x7f057012f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:27.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.391+0000 7f056effd640 1 --2- 172.21.15.102:0/2277931950 >> v2:172.21.15.102:3300/0 conn(0x7f057012f440 0x7f057012f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.391+0000 7f056effd640 1 -- 172.21.15.102:0/2277931950 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05700a22e0 con 0x7f0570097130 2024-09-13T17:44:27.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.391+0000 7f056e7fc640 1 --2- 172.21.15.102:0/2277931950 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f05700690e0 0x7f05700694d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:27.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.391+0000 7f056f7fe640 1 --2- 172.21.15.102:0/2277931950 >> v2:172.21.15.102:3300/0 conn(0x7f057012f440 0x7f057012f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:27.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.392+0000 7f056effd640 1 --2- 172.21.15.102:0/2277931950 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0570097130 0x7f0570068ba0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0560009910 tx=0x7f05600091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e047cc93127413af server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:27.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.392+0000 7f056dffb640 1 -- 172.21.15.102:0/2277931950 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056002ad20 con 0x7f0570097130 2024-09-13T17:44:27.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.392+0000 7f056dffb640 1 -- 172.21.15.102:0/2277931950 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f056002ae80 con 0x7f0570097130 2024-09-13T17:44:27.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.392+0000 7f056dffb640 1 -- 172.21.15.102:0/2277931950 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0560032670 con 0x7f0570097130 2024-09-13T17:44:27.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.392+0000 7f05756a8640 1 -- 172.21.15.102:0/2277931950 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0570097130 msgr2=0x7f0570068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.392+0000 7f05756a8640 1 --2- 172.21.15.102:0/2277931950 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0570097130 0x7f0570068ba0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0560009910 tx=0x7f05600091d0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.392+0000 7f05756a8640 1 -- 172.21.15.102:0/2277931950 shutdown_connections 2024-09-13T17:44:27.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.392+0000 7f05756a8640 1 --2- 172.21.15.102:0/2277931950 >> v2:172.21.15.102:3300/0 conn(0x7f057012f440 0x7f057012f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.392+0000 7f05756a8640 1 --2- 172.21.15.102:0/2277931950 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f05700690e0 0x7f05700694d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.392+0000 7f05756a8640 1 --2- 172.21.15.102:0/2277931950 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f0570097130 0x7f0570068ba0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.392+0000 7f05756a8640 1 -- 172.21.15.102:0/2277931950 >> 172.21.15.102:0/2277931950 conn(0x7f05700a34c0 msgr2=0x7f057006e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:27.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.392+0000 7f05756a8640 1 -- 172.21.15.102:0/2277931950 shutdown_connections 2024-09-13T17:44:27.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.392+0000 7f05756a8640 1 -- 172.21.15.102:0/2277931950 wait complete. 2024-09-13T17:44:27.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.393+0000 7f05756a8640 1 Processor -- start 2024-09-13T17:44:27.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.393+0000 7f05756a8640 1 -- start start 2024-09-13T17:44:27.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.394+0000 7f05756a8640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05700690e0 0x7f057012e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.394+0000 7f05756a8640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0570097130 0x7f057011a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.394+0000 7f05756a8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f057012f440 0x7f057011aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.394+0000 7f05756a8640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057011dee0 con 0x7f057012f440 2024-09-13T17:44:27.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.394+0000 7f05756a8640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f057011e0c0 con 0x7f05700690e0 2024-09-13T17:44:27.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.394+0000 7f05756a8640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f057009fc50 con 0x7f0570097130 2024-09-13T17:44:27.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.394+0000 7f056effd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05700690e0 0x7f057012e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.394+0000 7f056effd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05700690e0 0x7f057012e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48838/0 (socket says 172.21.15.102:48838) 2024-09-13T17:44:27.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.394+0000 7f056e7fc640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0570097130 0x7f057011a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.394+0000 7f056effd640 1 -- 172.21.15.102:0/3664577284 learned_addr learned my addr 172.21.15.102:0/3664577284 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:27.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.394+0000 7f056f7fe640 1 --2- 172.21.15.102:0/3664577284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f057012f440 0x7f057011aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.394+0000 7f056e7fc640 1 -- 172.21.15.102:0/3664577284 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05700690e0 msgr2=0x7f057012e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.394+0000 7f056e7fc640 1 --2- 172.21.15.102:0/3664577284 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05700690e0 0x7f057012e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.394+0000 7f056e7fc640 1 -- 172.21.15.102:0/3664577284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f057012f440 msgr2=0x7f057011aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.394+0000 7f056e7fc640 1 --2- 172.21.15.102:0/3664577284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f057012f440 0x7f057011aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.394+0000 7f056e7fc640 1 -- 172.21.15.102:0/3664577284 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0564009390 con 0x7f0570097130 2024-09-13T17:44:27.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.394+0000 7f056effd640 1 --2- 172.21.15.102:0/3664577284 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f05700690e0 0x7f057012e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:27.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.394+0000 7f056e7fc640 1 --2- 172.21.15.102:0/3664577284 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0570097130 0x7f057011a9a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f056400ea50 tx=0x7f05640090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:27.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.394+0000 7f0556ffd640 1 -- 172.21.15.102:0/3664577284 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056400c940 con 0x7f0570097130 2024-09-13T17:44:27.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.395+0000 7f0556ffd640 1 -- 172.21.15.102:0/3664577284 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f056400caa0 con 0x7f0570097130 2024-09-13T17:44:27.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.395+0000 7f0556ffd640 1 -- 172.21.15.102:0/3664577284 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056400ce40 con 0x7f0570097130 2024-09-13T17:44:27.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.395+0000 7f05756a8640 1 -- 172.21.15.102:0/3664577284 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0568009a20 con 0x7f0570097130 2024-09-13T17:44:27.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.395+0000 7f05756a8640 1 -- 172.21.15.102:0/3664577284 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0560009450 con 0x7f0570097130 2024-09-13T17:44:27.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.395+0000 7f0556ffd640 1 -- 172.21.15.102:0/3664577284 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f0564013680 con 0x7f0570097130 2024-09-13T17:44:27.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.395+0000 7f05756a8640 1 -- 172.21.15.102:0/3664577284 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f057011c580 con 0x7f0570097130 2024-09-13T17:44:27.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.396+0000 7f0556ffd640 1 -- 172.21.15.102:0/3664577284 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 2851+0+0 (secure 0 0 0) 0x7f0564041fd0 con 0x7f0570097130 2024-09-13T17:44:27.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.396+0000 7f0556ffd640 1 -- 172.21.15.102:0/3664577284 --> v1:172.21.15.102:6804/35328 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f055c03b5b0 con 0x7f055c037910 2024-09-13T17:44:27.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.396+0000 7f0556ffd640 1 -- 172.21.15.102:0/3664577284 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05640445a0 con 0x7f0570097130 2024-09-13T17:44:27.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.397+0000 7f0556ffd640 1 -- 172.21.15.102:0/3664577284 <== osd.1 v1:172.21.15.102:6804/35328 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f055c03b5b0 con 0x7f055c037910 2024-09-13T17:44:27.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.408+0000 7f05756a8640 1 -- 172.21.15.102:0/3664577284 --> v1:172.21.15.102:6804/35328 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f057011e200 con 0x7f055c037910 2024-09-13T17:44:27.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.408+0000 7f0556ffd640 1 -- 172.21.15.102:0/3664577284 <== osd.1 v1:172.21.15.102:6804/35328 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (unknown 0 0 119112422) 0x7f057011e200 con 0x7f055c037910 2024-09-13T17:44:27.411 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-13T17:44:27.411 INFO:teuthology.orchestra.run.smithi102.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T17:44:27.411 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-13T17:44:27.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.408+0000 7f05756a8640 1 -- 172.21.15.102:0/3664577284 >> v1:172.21.15.102:6804/35328 conn(0x7f055c037910 legacy=0x7f055c039d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.408+0000 7f05756a8640 1 -- 172.21.15.102:0/3664577284 >> v1:172.21.15.102:6816/34937 conn(0x7f055c031160 legacy=0x7f055c033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.408+0000 7f05756a8640 1 -- 172.21.15.102:0/3664577284 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0570097130 msgr2=0x7f057011a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.408+0000 7f05756a8640 1 --2- 172.21.15.102:0/3664577284 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f0570097130 0x7f057011a9a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f056400ea50 tx=0x7f05640090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.408+0000 7f056f7fe640 1 -- 172.21.15.102:0/3664577284 reap_dead start 2024-09-13T17:44:27.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.409+0000 7f05756a8640 1 -- 172.21.15.102:0/3664577284 shutdown_connections 2024-09-13T17:44:27.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.409+0000 7f05756a8640 1 -- 172.21.15.102:0/3664577284 >> 172.21.15.102:0/3664577284 conn(0x7f05700a34c0 msgr2=0x7f0570132510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:27.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.409+0000 7f05756a8640 1 -- 172.21.15.102:0/3664577284 shutdown_connections 2024-09-13T17:44:27.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.409+0000 7f05756a8640 1 -- 172.21.15.102:0/3664577284 wait complete. 2024-09-13T17:44:27.421 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-13T17:44:27.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.513+0000 7facadb54640 1 Processor -- start 2024-09-13T17:44:27.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.513+0000 7facadb54640 1 -- start start 2024-09-13T17:44:27.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.514+0000 7facadb54640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faca812c870 0x7faca8131490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.514+0000 7facadb54640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faca812cc80 0x7faca81339e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.514+0000 7facadb54640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faca806a530 con 0x7faca809abb0 2024-09-13T17:44:27.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.514+0000 7facadb54640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca809af80 con 0x7faca812c870 2024-09-13T17:44:27.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.514+0000 7facadb54640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faca8098be0 con 0x7faca812cc80 2024-09-13T17:44:27.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.514+0000 7faca6ffd640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faca812c870 0x7faca8131490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.514+0000 7faca7fff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faca812cc80 0x7faca81339e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.514+0000 7faca77fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7faca809abb0 0x7faca812c160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:35894/0 (socket says 172.21.15.102:35894) 2024-09-13T17:44:27.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.514+0000 7faca77fe640 1 -- 172.21.15.102:0/1784191050 learned_addr learned my addr 172.21.15.102:0/1784191050 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:44:27.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.514+0000 7faca6ffd640 1 -- 172.21.15.102:0/1784191050 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faca812cc80 msgr2=0x7faca81339e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:27.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.514+0000 7faca67fc640 1 -- 172.21.15.102:0/1784191050 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 542635721 0 0) 0x7faca806a530 con 0x7faca809abb0 2024-09-13T17:44:27.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.514+0000 7faca6ffd640 1 --2- 172.21.15.102:0/1784191050 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faca812cc80 0x7faca81339e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.514+0000 7faca6ffd640 1 -- 172.21.15.102:0/1784191050 >> v1:172.21.15.102:6789/0 conn(0x7faca809abb0 legacy=0x7faca812c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.514+0000 7faca6ffd640 1 -- 172.21.15.102:0/1784191050 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca8098930 con 0x7faca812c870 2024-09-13T17:44:27.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.514+0000 7faca7fff640 1 --2- 172.21.15.102:0/1784191050 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faca812cc80 0x7faca81339e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:27.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.515+0000 7faca6ffd640 1 --2- 172.21.15.102:0/1784191050 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faca812c870 0x7faca8131490 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fac9c00bb20 tx=0x7fac9c00b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=306ec23e2d6a94e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:27.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.515+0000 7faca67fc640 1 -- 172.21.15.102:0/1784191050 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c031510 con 0x7faca812c870 2024-09-13T17:44:27.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.515+0000 7faca67fc640 1 -- 172.21.15.102:0/1784191050 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9c02f050 con 0x7faca812c870 2024-09-13T17:44:27.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.515+0000 7faca67fc640 1 -- 172.21.15.102:0/1784191050 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c0318c0 con 0x7faca812c870 2024-09-13T17:44:27.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.515+0000 7facadb54640 1 -- 172.21.15.102:0/1784191050 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faca812c870 msgr2=0x7faca8131490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.515+0000 7facadb54640 1 --2- 172.21.15.102:0/1784191050 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faca812c870 0x7faca8131490 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fac9c00bb20 tx=0x7fac9c00b3b0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.515+0000 7facadb54640 1 -- 172.21.15.102:0/1784191050 shutdown_connections 2024-09-13T17:44:27.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.516+0000 7facadb54640 1 --2- 172.21.15.102:0/1784191050 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faca812cc80 0x7faca81339e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.516+0000 7facadb54640 1 --2- 172.21.15.102:0/1784191050 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faca812c870 0x7faca8131490 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.516+0000 7facadb54640 1 -- 172.21.15.102:0/1784191050 >> 172.21.15.102:0/1784191050 conn(0x7faca809e820 msgr2=0x7faca809ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:27.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.516+0000 7facadb54640 1 -- 172.21.15.102:0/1784191050 shutdown_connections 2024-09-13T17:44:27.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.516+0000 7facadb54640 1 -- 172.21.15.102:0/1784191050 wait complete. 2024-09-13T17:44:27.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.516+0000 7facadb54640 1 Processor -- start 2024-09-13T17:44:27.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.517+0000 7facadb54640 1 -- start start 2024-09-13T17:44:27.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.517+0000 7facadb54640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faca809abb0 0x7faca813a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.517+0000 7facadb54640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faca812c870 0x7faca813ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.517+0000 7facadb54640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faca812cc80 0x7faca81152c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.517+0000 7facadb54640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca8118980 con 0x7faca812cc80 2024-09-13T17:44:27.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.517+0000 7facadb54640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca812f490 con 0x7faca809abb0 2024-09-13T17:44:27.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.517+0000 7facadb54640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faca812a3b0 con 0x7faca812c870 2024-09-13T17:44:27.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.517+0000 7faca77fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faca809abb0 0x7faca813a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.517+0000 7faca6ffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faca812c870 0x7faca813ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.517+0000 7faca77fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faca809abb0 0x7faca813a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48850/0 (socket says 172.21.15.102:48850) 2024-09-13T17:44:27.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.517+0000 7faca77fe640 1 -- 172.21.15.102:0/2989934490 learned_addr learned my addr 172.21.15.102:0/2989934490 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:27.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.517+0000 7faca6ffd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faca812c870 0x7faca813ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:42222/0 (socket says 172.21.15.102:42222) 2024-09-13T17:44:27.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.517+0000 7faca7fff640 1 --2- 172.21.15.102:0/2989934490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faca812cc80 0x7faca81152c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.517+0000 7faca6ffd640 1 -- 172.21.15.102:0/2989934490 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faca809abb0 msgr2=0x7faca813a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.517+0000 7faca6ffd640 1 --2- 172.21.15.102:0/2989934490 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faca809abb0 0x7faca813a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.517+0000 7faca6ffd640 1 -- 172.21.15.102:0/2989934490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faca812cc80 msgr2=0x7faca81152c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.517+0000 7faca6ffd640 1 --2- 172.21.15.102:0/2989934490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faca812cc80 0x7faca81152c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.517+0000 7faca6ffd640 1 -- 172.21.15.102:0/2989934490 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca0009a20 con 0x7faca812c870 2024-09-13T17:44:27.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.517+0000 7faca77fe640 1 --2- 172.21.15.102:0/2989934490 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faca809abb0 0x7faca813a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:27.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.518+0000 7faca6ffd640 1 --2- 172.21.15.102:0/2989934490 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faca812c870 0x7faca813ac80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fac9c033000 tx=0x7fac9c039000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:27.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.518+0000 7fac8f7fe640 1 -- 172.21.15.102:0/2989934490 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c03dd80 con 0x7faca812c870 2024-09-13T17:44:27.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.518+0000 7fac8f7fe640 1 -- 172.21.15.102:0/2989934490 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9c047420 con 0x7faca812c870 2024-09-13T17:44:27.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.518+0000 7facadb54640 1 -- 172.21.15.102:0/2989934490 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac9c00b630 con 0x7faca812c870 2024-09-13T17:44:27.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.518+0000 7fac8f7fe640 1 -- 172.21.15.102:0/2989934490 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c047820 con 0x7faca812c870 2024-09-13T17:44:27.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.518+0000 7facadb54640 1 -- 172.21.15.102:0/2989934490 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca81193b0 con 0x7faca812c870 2024-09-13T17:44:27.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.518+0000 7fac8f7fe640 1 -- 172.21.15.102:0/2989934490 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fac9c033400 con 0x7faca812c870 2024-09-13T17:44:27.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.519+0000 7fac8f7fe640 1 -- 172.21.15.102:0/2989934490 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 2851+0+0 (secure 0 0 0) 0x7fac9c066f90 con 0x7faca812c870 2024-09-13T17:44:27.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.519+0000 7facadb54640 1 -- 172.21.15.102:0/2989934490 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faca8116730 con 0x7faca812c870 2024-09-13T17:44:27.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.523+0000 7fac8f7fe640 1 -- 172.21.15.102:0/2989934490 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fac9c032700 con 0x7faca812c870 2024-09-13T17:44:27.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.677+0000 7facadb54640 1 -- 172.21.15.102:0/2989934490 --> v1:172.21.15.102:6816/34937 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7faca8118ac0 con 0x7fac94031160 2024-09-13T17:44:27.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.677+0000 7fac8f7fe640 1 -- 172.21.15.102:0/2989934490 <== mgr.4101 v1:172.21.15.102:6816/34937 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (unknown 489605429 0 0) 0x7faca8118ac0 con 0x7fac94031160 2024-09-13T17:44:27.679 INFO:teuthology.orchestra.run.smithi102.stderr:instructed osd(s) 1 to deep-scrub 2024-09-13T17:44:27.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.679+0000 7facadb54640 1 -- 172.21.15.102:0/2989934490 >> v1:172.21.15.102:6816/34937 conn(0x7fac94031160 legacy=0x7fac94033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.679+0000 7facadb54640 1 -- 172.21.15.102:0/2989934490 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faca812c870 msgr2=0x7faca813ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.679+0000 7facadb54640 1 --2- 172.21.15.102:0/2989934490 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faca812c870 0x7faca813ac80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fac9c033000 tx=0x7fac9c039000 comp rx=0 tx=0).stop 2024-09-13T17:44:27.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.679+0000 7facadb54640 1 -- 172.21.15.102:0/2989934490 shutdown_connections 2024-09-13T17:44:27.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.679+0000 7facadb54640 1 --2- 172.21.15.102:0/2989934490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faca812cc80 0x7faca81152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.679+0000 7facadb54640 1 --2- 172.21.15.102:0/2989934490 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7faca812c870 0x7faca813ac80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.679+0000 7facadb54640 1 --2- 172.21.15.102:0/2989934490 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7faca809abb0 0x7faca813a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.679+0000 7facadb54640 1 -- 172.21.15.102:0/2989934490 >> 172.21.15.102:0/2989934490 conn(0x7faca809e820 msgr2=0x7faca809b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:27.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.679+0000 7facadb54640 1 -- 172.21.15.102:0/2989934490 shutdown_connections 2024-09-13T17:44:27.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.679+0000 7facadb54640 1 -- 172.21.15.102:0/2989934490 wait complete. 2024-09-13T17:44:27.691 INFO:tasks.ceph:Scrubbing osd.2 2024-09-13T17:44:27.692 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-13T17:44:27.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.786+0000 7f67ccec0640 1 Processor -- start 2024-09-13T17:44:27.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.786+0000 7f67ccec0640 1 -- start start 2024-09-13T17:44:27.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.787+0000 7f67ccec0640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f67c8096a70 0x7f67c812c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.787+0000 7f67ccec0640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f67c812cac0 0x7f67c8098410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.787+0000 7f67ccec0640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c809ad20 con 0x7f67c812ced0 2024-09-13T17:44:27.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.787+0000 7f67ccec0640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67c8096e40 con 0x7f67c812cac0 2024-09-13T17:44:27.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.787+0000 7f67ccec0640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67c80996e0 con 0x7f67c8096a70 2024-09-13T17:44:27.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.787+0000 7f67c5d9b640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f67c812cac0 0x7f67c8098410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.787+0000 7f67c659c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f67c8096a70 0x7f67c812c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.787+0000 7f67c6d9d640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f67c812ced0 0x7f67c8098950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:35908/0 (socket says 172.21.15.102:35908) 2024-09-13T17:44:27.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.787+0000 7f67c6d9d640 1 -- 172.21.15.102:0/107746406 learned_addr learned my addr 172.21.15.102:0/107746406 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:44:27.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.787+0000 7f67c5d9b640 1 -- 172.21.15.102:0/107746406 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f67c8096a70 msgr2=0x7f67c812c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:27.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.787+0000 7f67c5d9b640 1 --2- 172.21.15.102:0/107746406 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f67c8096a70 0x7f67c812c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.787+0000 7f67c5d9b640 1 -- 172.21.15.102:0/107746406 >> v1:172.21.15.102:6789/0 conn(0x7f67c812ced0 legacy=0x7f67c8098950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.787+0000 7f67c5d9b640 1 -- 172.21.15.102:0/107746406 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c809aa60 con 0x7f67c812cac0 2024-09-13T17:44:27.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.787+0000 7f67c559a640 1 -- 172.21.15.102:0/107746406 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3126387716 0 0) 0x7f67c809ad20 con 0x7f67c812ced0 2024-09-13T17:44:27.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.787+0000 7f67c659c640 1 --2- 172.21.15.102:0/107746406 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f67c8096a70 0x7f67c812c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:27.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.788+0000 7f67c5d9b640 1 --2- 172.21.15.102:0/107746406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67c812cac0 0x7f67c8098410 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f67b0009880 tx=0x7f67b00090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e28f8f0b73b5b4b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:27.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.788+0000 7f67c559a640 1 -- 172.21.15.102:0/107746406 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b002ac50 con 0x7f67c812cac0 2024-09-13T17:44:27.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.788+0000 7f67c559a640 1 -- 172.21.15.102:0/107746406 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b002adb0 con 0x7f67c812cac0 2024-09-13T17:44:27.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.788+0000 7f67c559a640 1 -- 172.21.15.102:0/107746406 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b0032670 con 0x7f67c812cac0 2024-09-13T17:44:27.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.789+0000 7f67ccec0640 1 -- 172.21.15.102:0/107746406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67c812cac0 msgr2=0x7f67c8098410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.789+0000 7f67ccec0640 1 --2- 172.21.15.102:0/107746406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67c812cac0 0x7f67c8098410 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f67b0009880 tx=0x7f67b00090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.789+0000 7f67ccec0640 1 -- 172.21.15.102:0/107746406 shutdown_connections 2024-09-13T17:44:27.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.789+0000 7f67ccec0640 1 --2- 172.21.15.102:0/107746406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67c812cac0 0x7f67c8098410 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.789+0000 7f67ccec0640 1 --2- 172.21.15.102:0/107746406 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f67c8096a70 0x7f67c812c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.789+0000 7f67ccec0640 1 -- 172.21.15.102:0/107746406 >> 172.21.15.102:0/107746406 conn(0x7f67c809e720 msgr2=0x7f67c809eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:27.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.789+0000 7f67ccec0640 1 -- 172.21.15.102:0/107746406 shutdown_connections 2024-09-13T17:44:27.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.789+0000 7f67ccec0640 1 -- 172.21.15.102:0/107746406 wait complete. 2024-09-13T17:44:27.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.790+0000 7f67ccec0640 1 Processor -- start 2024-09-13T17:44:27.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.790+0000 7f67ccec0640 1 -- start start 2024-09-13T17:44:27.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.790+0000 7f67ccec0640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67c8096a70 0x7f67c81296d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.790+0000 7f67ccec0640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f67c812cac0 0x7f67c8118440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.790+0000 7f67ccec0640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67c812ced0 0x7f67c8118980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.790+0000 7f67ccec0640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c809a330 con 0x7f67c8096a70 2024-09-13T17:44:27.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.790+0000 7f67ccec0640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c8097e40 con 0x7f67c812ced0 2024-09-13T17:44:27.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.790+0000 7f67ccec0640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67c811ee70 con 0x7f67c812cac0 2024-09-13T17:44:27.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.790+0000 7f67c659c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67c8096a70 0x7f67c81296d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.790+0000 7f67c5d9b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f67c812cac0 0x7f67c8118440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.790+0000 7f67c6d9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67c812ced0 0x7f67c8118980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.790+0000 7f67c5d9b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f67c812cac0 0x7f67c8118440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:42248/0 (socket says 172.21.15.102:42248) 2024-09-13T17:44:27.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.790+0000 7f67c5d9b640 1 -- 172.21.15.102:0/2668274956 learned_addr learned my addr 172.21.15.102:0/2668274956 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:27.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.790+0000 7f67c659c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67c8096a70 0x7f67c81296d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55756/0 (socket says 172.21.15.102:55756) 2024-09-13T17:44:27.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.791+0000 7f67c5d9b640 1 -- 172.21.15.102:0/2668274956 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67c812ced0 msgr2=0x7f67c8118980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.791+0000 7f67c5d9b640 1 --2- 172.21.15.102:0/2668274956 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f67c812ced0 0x7f67c8118980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.791+0000 7f67c5d9b640 1 -- 172.21.15.102:0/2668274956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67c8096a70 msgr2=0x7f67c81296d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.791+0000 7f67c5d9b640 1 --2- 172.21.15.102:0/2668274956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67c8096a70 0x7f67c81296d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.791+0000 7f67c5d9b640 1 -- 172.21.15.102:0/2668274956 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b8009390 con 0x7f67c812cac0 2024-09-13T17:44:27.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.791+0000 7f67c659c640 1 --2- 172.21.15.102:0/2668274956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67c8096a70 0x7f67c81296d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:27.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.791+0000 7f67c5d9b640 1 --2- 172.21.15.102:0/2668274956 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f67c812cac0 0x7f67c8118440 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f67b0009360 tx=0x7f67b0032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:27.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.791+0000 7f67be7fc640 1 -- 172.21.15.102:0/2668274956 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b0031780 con 0x7f67c812cac0 2024-09-13T17:44:27.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.791+0000 7f67be7fc640 1 -- 172.21.15.102:0/2668274956 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b00318e0 con 0x7f67c812cac0 2024-09-13T17:44:27.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.791+0000 7f67be7fc640 1 -- 172.21.15.102:0/2668274956 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b0031ca0 con 0x7f67c812cac0 2024-09-13T17:44:27.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.791+0000 7f67ccec0640 1 -- 172.21.15.102:0/2668274956 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b0009390 con 0x7f67c812cac0 2024-09-13T17:44:27.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.791+0000 7f67ccec0640 1 -- 172.21.15.102:0/2668274956 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c8129c10 con 0x7f67c812cac0 2024-09-13T17:44:27.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.792+0000 7f67be7fc640 1 -- 172.21.15.102:0/2668274956 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f67b0039040 con 0x7f67c812cac0 2024-09-13T17:44:27.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.792+0000 7f67ccec0640 1 -- 172.21.15.102:0/2668274956 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f67c8064000 con 0x7f67c812cac0 2024-09-13T17:44:27.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.792+0000 7f67be7fc640 1 -- 172.21.15.102:0/2668274956 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 2851+0+0 (secure 0 0 0) 0x7f67b0061650 con 0x7f67c812cac0 2024-09-13T17:44:27.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.792+0000 7f67be7fc640 1 -- 172.21.15.102:0/2668274956 --> v1:172.21.15.102:6808/35351 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f67a803b5b0 con 0x7f67a8037910 2024-09-13T17:44:27.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.792+0000 7f67be7fc640 1 -- 172.21.15.102:0/2668274956 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67b003baf0 con 0x7f67c812cac0 2024-09-13T17:44:27.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.793+0000 7f67be7fc640 1 -- 172.21.15.102:0/2668274956 <== osd.2 v1:172.21.15.102:6808/35351 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f67a803b5b0 con 0x7f67a8037910 2024-09-13T17:44:27.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.804+0000 7f67ccec0640 1 -- 172.21.15.102:0/2668274956 --> v1:172.21.15.102:6808/35351 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f67c80975c0 con 0x7f67a8037910 2024-09-13T17:44:27.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.804+0000 7f67be7fc640 1 -- 172.21.15.102:0/2668274956 <== osd.2 v1:172.21.15.102:6808/35351 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (unknown 0 0 119112422) 0x7f67c80975c0 con 0x7f67a8037910 2024-09-13T17:44:27.806 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-13T17:44:27.807 INFO:teuthology.orchestra.run.smithi102.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T17:44:27.807 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-13T17:44:27.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.805+0000 7f67ccec0640 1 -- 172.21.15.102:0/2668274956 >> v1:172.21.15.102:6808/35351 conn(0x7f67a8037910 legacy=0x7f67a8039d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.805+0000 7f67ccec0640 1 -- 172.21.15.102:0/2668274956 >> v1:172.21.15.102:6816/34937 conn(0x7f67a8031160 legacy=0x7f67a8033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.805+0000 7f67ccec0640 1 -- 172.21.15.102:0/2668274956 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f67c812cac0 msgr2=0x7f67c8118440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.805+0000 7f67ccec0640 1 --2- 172.21.15.102:0/2668274956 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f67c812cac0 0x7f67c8118440 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f67b0009360 tx=0x7f67b0032000 comp rx=0 tx=0).stop 2024-09-13T17:44:27.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.805+0000 7f67c6d9d640 1 -- 172.21.15.102:0/2668274956 reap_dead start 2024-09-13T17:44:27.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.805+0000 7f67ccec0640 1 -- 172.21.15.102:0/2668274956 shutdown_connections 2024-09-13T17:44:27.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.805+0000 7f67ccec0640 1 -- 172.21.15.102:0/2668274956 >> 172.21.15.102:0/2668274956 conn(0x7f67c809e720 msgr2=0x7f67c809d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:27.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.805+0000 7f67ccec0640 1 -- 172.21.15.102:0/2668274956 shutdown_connections 2024-09-13T17:44:27.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.805+0000 7f67ccec0640 1 -- 172.21.15.102:0/2668274956 wait complete. 2024-09-13T17:44:27.816 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-13T17:44:27.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.908+0000 7f8685657640 1 Processor -- start 2024-09-13T17:44:27.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.908+0000 7f8685657640 1 -- start start 2024-09-13T17:44:27.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.909+0000 7f8685657640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8680128e10 0x7f8680130b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.909+0000 7f8685657640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f86801310b0 0x7f8680133490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.909+0000 7f8685657640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8680097be0 con 0x7f8680128330 2024-09-13T17:44:27.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.909+0000 7f8685657640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8680129200 con 0x7f86801310b0 2024-09-13T17:44:27.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.909+0000 7f8685657640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8680098e70 con 0x7f8680128e10 2024-09-13T17:44:27.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.909+0000 7f86767fc640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8680128e10 0x7f8680130b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.909+0000 7f867f7fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f86801310b0 0x7f8680133490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.909+0000 7f867effd640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f8680128330 0x7f8680128700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:35924/0 (socket says 172.21.15.102:35924) 2024-09-13T17:44:27.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.909+0000 7f867effd640 1 -- 172.21.15.102:0/3382452078 learned_addr learned my addr 172.21.15.102:0/3382452078 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:44:27.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.909+0000 7f867e7fc640 1 -- 172.21.15.102:0/3382452078 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3364550958 0 0) 0x7f8680097be0 con 0x7f8680128330 2024-09-13T17:44:27.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.909+0000 7f867f7fe640 1 -- 172.21.15.102:0/3382452078 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8680128e10 msgr2=0x7f8680130b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:27.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.909+0000 7f867f7fe640 1 --2- 172.21.15.102:0/3382452078 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8680128e10 0x7f8680130b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.909+0000 7f867f7fe640 1 -- 172.21.15.102:0/3382452078 >> v1:172.21.15.102:6789/0 conn(0x7f8680128330 legacy=0x7f8680128700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.909+0000 7f867f7fe640 1 -- 172.21.15.102:0/3382452078 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8680098310 con 0x7f86801310b0 2024-09-13T17:44:27.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.909+0000 7f86767fc640 1 --2- 172.21.15.102:0/3382452078 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8680128e10 0x7f8680130b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:27.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.910+0000 7f867f7fe640 1 --2- 172.21.15.102:0/3382452078 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f86801310b0 0x7f8680133490 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8678009e90 tx=0x7f8678009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f25b1124a8415a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:27.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.910+0000 7f867e7fc640 1 -- 172.21.15.102:0/3382452078 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8678007550 con 0x7f86801310b0 2024-09-13T17:44:27.915 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.910+0000 7f867e7fc640 1 -- 172.21.15.102:0/3382452078 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867802d050 con 0x7f86801310b0 2024-09-13T17:44:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.910+0000 7f867e7fc640 1 -- 172.21.15.102:0/3382452078 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86780078e0 con 0x7f86801310b0 2024-09-13T17:44:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.910+0000 7f8685657640 1 -- 172.21.15.102:0/3382452078 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f86801310b0 msgr2=0x7f8680133490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.910+0000 7f8685657640 1 --2- 172.21.15.102:0/3382452078 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f86801310b0 0x7f8680133490 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8678009e90 tx=0x7f8678009780 comp rx=0 tx=0).stop 2024-09-13T17:44:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.911+0000 7f8685657640 1 -- 172.21.15.102:0/3382452078 shutdown_connections 2024-09-13T17:44:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.911+0000 7f8685657640 1 --2- 172.21.15.102:0/3382452078 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f86801310b0 0x7f8680133490 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.911+0000 7f8685657640 1 --2- 172.21.15.102:0/3382452078 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8680128e10 0x7f8680130b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.911+0000 7f8685657640 1 -- 172.21.15.102:0/3382452078 >> 172.21.15.102:0/3382452078 conn(0x7f868009e820 msgr2=0x7f868009ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.911+0000 7f8685657640 1 -- 172.21.15.102:0/3382452078 shutdown_connections 2024-09-13T17:44:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.911+0000 7f8685657640 1 -- 172.21.15.102:0/3382452078 wait complete. 2024-09-13T17:44:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.911+0000 7f8685657640 1 Processor -- start 2024-09-13T17:44:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.911+0000 7f8685657640 1 -- start start 2024-09-13T17:44:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.912+0000 7f8685657640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8680128330 0x7f868013c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.912+0000 7f8685657640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8680128e10 0x7f868013cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.912+0000 7f8685657640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86801310b0 0x7f868006c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.912+0000 7f8685657640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868006d1c0 con 0x7f86801310b0 2024-09-13T17:44:27.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.912+0000 7f8685657640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f868006ebc0 con 0x7f8680128e10 2024-09-13T17:44:27.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.912+0000 7f8685657640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8680125f00 con 0x7f8680128330 2024-09-13T17:44:27.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.912+0000 7f867effd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8680128330 0x7f868013c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.912+0000 7f867effd640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8680128330 0x7f868013c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:42270/0 (socket says 172.21.15.102:42270) 2024-09-13T17:44:27.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.912+0000 7f867effd640 1 -- 172.21.15.102:0/347048574 learned_addr learned my addr 172.21.15.102:0/347048574 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:27.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.912+0000 7f867f7fe640 1 --2- 172.21.15.102:0/347048574 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86801310b0 0x7f868006c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.912+0000 7f86767fc640 1 --2- 172.21.15.102:0/347048574 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8680128e10 0x7f868013cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.912+0000 7f867f7fe640 1 -- 172.21.15.102:0/347048574 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8680128330 msgr2=0x7f868013c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.912+0000 7f867f7fe640 1 --2- 172.21.15.102:0/347048574 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8680128330 0x7f868013c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.912+0000 7f867f7fe640 1 -- 172.21.15.102:0/347048574 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8680128e10 msgr2=0x7f868013cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.912+0000 7f867f7fe640 1 --2- 172.21.15.102:0/347048574 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8680128e10 0x7f868013cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.912+0000 7f867f7fe640 1 -- 172.21.15.102:0/347048574 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866800a530 con 0x7f86801310b0 2024-09-13T17:44:27.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.912+0000 7f867effd640 1 --2- 172.21.15.102:0/347048574 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8680128330 0x7f868013c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:27.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.912+0000 7f86767fc640 1 --2- 172.21.15.102:0/347048574 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8680128e10 0x7f868013cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:27.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.912+0000 7f867f7fe640 1 --2- 172.21.15.102:0/347048574 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86801310b0 0x7f868006c0d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8678031000 tx=0x7f8678037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:27.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.913+0000 7f86777fe640 1 -- 172.21.15.102:0/347048574 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8678031890 con 0x7f86801310b0 2024-09-13T17:44:27.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.913+0000 7f86777fe640 1 -- 172.21.15.102:0/347048574 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8678040070 con 0x7f86801310b0 2024-09-13T17:44:27.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.913+0000 7f8685657640 1 -- 172.21.15.102:0/347048574 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8678009a20 con 0x7f86801310b0 2024-09-13T17:44:27.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.913+0000 7f86777fe640 1 -- 172.21.15.102:0/347048574 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8678031c80 con 0x7f86801310b0 2024-09-13T17:44:27.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.913+0000 7f8685657640 1 -- 172.21.15.102:0/347048574 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f868012ff50 con 0x7f86801310b0 2024-09-13T17:44:27.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.913+0000 7f86777fe640 1 -- 172.21.15.102:0/347048574 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f8678042030 con 0x7f86801310b0 2024-09-13T17:44:27.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.914+0000 7f86777fe640 1 -- 172.21.15.102:0/347048574 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 2851+0+0 (secure 0 0 0) 0x7f8678063650 con 0x7f86801310b0 2024-09-13T17:44:27.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.914+0000 7f8685657640 1 -- 172.21.15.102:0/347048574 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8680064000 con 0x7f86801310b0 2024-09-13T17:44:27.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:27.918+0000 7f86777fe640 1 -- 172.21.15.102:0/347048574 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f867803c5e0 con 0x7f86801310b0 2024-09-13T17:44:28.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.069+0000 7f8685657640 1 -- 172.21.15.102:0/347048574 --> v1:172.21.15.102:6816/34937 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f8680067690 con 0x7f8660031160 2024-09-13T17:44:28.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.070+0000 7f86777fe640 1 -- 172.21.15.102:0/347048574 <== mgr.4101 v1:172.21.15.102:6816/34937 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (unknown 3049115190 0 0) 0x7f8680067690 con 0x7f8660031160 2024-09-13T17:44:28.072 INFO:teuthology.orchestra.run.smithi102.stderr:instructed osd(s) 2 to deep-scrub 2024-09-13T17:44:28.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.071+0000 7f8685657640 1 -- 172.21.15.102:0/347048574 >> v1:172.21.15.102:6816/34937 conn(0x7f8660031160 legacy=0x7f8660033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:28.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.071+0000 7f8685657640 1 -- 172.21.15.102:0/347048574 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86801310b0 msgr2=0x7f868006c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:28.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.071+0000 7f8685657640 1 --2- 172.21.15.102:0/347048574 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86801310b0 0x7f868006c0d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8678031000 tx=0x7f8678037000 comp rx=0 tx=0).stop 2024-09-13T17:44:28.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.071+0000 7f8685657640 1 -- 172.21.15.102:0/347048574 shutdown_connections 2024-09-13T17:44:28.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.071+0000 7f8685657640 1 --2- 172.21.15.102:0/347048574 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86801310b0 0x7f868006c0d0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:28.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.071+0000 7f8685657640 1 --2- 172.21.15.102:0/347048574 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f8680128e10 0x7f868013cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:28.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.071+0000 7f8685657640 1 --2- 172.21.15.102:0/347048574 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f8680128330 0x7f868013c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:28.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.072+0000 7f8685657640 1 -- 172.21.15.102:0/347048574 >> 172.21.15.102:0/347048574 conn(0x7f868009e820 msgr2=0x7f8680129740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:28.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.072+0000 7f8685657640 1 -- 172.21.15.102:0/347048574 shutdown_connections 2024-09-13T17:44:28.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.072+0000 7f8685657640 1 -- 172.21.15.102:0/347048574 wait complete. 2024-09-13T17:44:28.085 INFO:tasks.ceph:Scrubbing osd.3 2024-09-13T17:44:28.085 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-13T17:44:28.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.180+0000 7fba9feaf640 1 Processor -- start 2024-09-13T17:44:28.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.181+0000 7fba9feaf640 1 -- start start 2024-09-13T17:44:28.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.181+0000 7fba9feaf640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fba980977d0 0x7fba9809a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:28.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.181+0000 7fba9feaf640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fba981350d0 0x7fba981374b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:28.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.181+0000 7fba9feaf640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba98095f50 con 0x7fba980959a0 2024-09-13T17:44:28.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.181+0000 7fba9feaf640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba98096b60 con 0x7fba981350d0 2024-09-13T17:44:28.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.181+0000 7fba9feaf640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba9809c230 con 0x7fba980977d0 2024-09-13T17:44:28.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.181+0000 7fba9dc4b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fba980977d0 0x7fba9809a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:28.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.181+0000 7fba9d44a640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fba980959a0 0x7fba9809ae50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:35940/0 (socket says 172.21.15.102:35940) 2024-09-13T17:44:28.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.181+0000 7fba9d44a640 1 -- 172.21.15.102:0/3742775119 learned_addr learned my addr 172.21.15.102:0/3742775119 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:44:28.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.181+0000 7fba9e44c640 1 --2- 172.21.15.102:0/3742775119 >> v2:172.21.15.102:3300/0 conn(0x7fba981350d0 0x7fba981374b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:28.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.182+0000 7fba9e44c640 1 -- 172.21.15.102:0/3742775119 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fba980977d0 msgr2=0x7fba9809a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:28.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.182+0000 7fba9e44c640 1 --2- 172.21.15.102:0/3742775119 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fba980977d0 0x7fba9809a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:28.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.182+0000 7fba9e44c640 1 -- 172.21.15.102:0/3742775119 >> v1:172.21.15.102:6789/0 conn(0x7fba980959a0 legacy=0x7fba9809ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:28.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.182+0000 7fba9e44c640 1 -- 172.21.15.102:0/3742775119 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba98096da0 con 0x7fba981350d0 2024-09-13T17:44:28.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.182+0000 7fba9cc49640 1 -- 172.21.15.102:0/3742775119 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1335858721 0 0) 0x7fba98095f50 con 0x7fba980959a0 2024-09-13T17:44:28.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.182+0000 7fba9dc4b640 1 --2- 172.21.15.102:0/3742775119 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fba980977d0 0x7fba9809a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:28.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.182+0000 7fba9e44c640 1 --2- 172.21.15.102:0/3742775119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba981350d0 0x7fba981374b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fba90009e60 tx=0x7fba90009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd89ffca67cc663d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:28.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.182+0000 7fba9cc49640 1 -- 172.21.15.102:0/3742775119 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba90007550 con 0x7fba981350d0 2024-09-13T17:44:28.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.182+0000 7fba9cc49640 1 -- 172.21.15.102:0/3742775119 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba9002d050 con 0x7fba981350d0 2024-09-13T17:44:28.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.182+0000 7fba9cc49640 1 -- 172.21.15.102:0/3742775119 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba900078e0 con 0x7fba981350d0 2024-09-13T17:44:28.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.183+0000 7fba9feaf640 1 -- 172.21.15.102:0/3742775119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba981350d0 msgr2=0x7fba981374b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:28.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.183+0000 7fba9feaf640 1 --2- 172.21.15.102:0/3742775119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba981350d0 0x7fba981374b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fba90009e60 tx=0x7fba90009780 comp rx=0 tx=0).stop 2024-09-13T17:44:28.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.183+0000 7fba9feaf640 1 -- 172.21.15.102:0/3742775119 shutdown_connections 2024-09-13T17:44:28.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.183+0000 7fba9feaf640 1 --2- 172.21.15.102:0/3742775119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba981350d0 0x7fba981374b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:28.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.183+0000 7fba9feaf640 1 --2- 172.21.15.102:0/3742775119 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fba980977d0 0x7fba9809a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:28.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.183+0000 7fba9feaf640 1 -- 172.21.15.102:0/3742775119 >> 172.21.15.102:0/3742775119 conn(0x7fba9809f930 msgr2=0x7fba9809ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:28.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.183+0000 7fba9feaf640 1 -- 172.21.15.102:0/3742775119 shutdown_connections 2024-09-13T17:44:28.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.183+0000 7fba9feaf640 1 -- 172.21.15.102:0/3742775119 wait complete. 2024-09-13T17:44:28.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.184+0000 7fba9feaf640 1 Processor -- start 2024-09-13T17:44:28.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.184+0000 7fba9feaf640 1 -- start start 2024-09-13T17:44:28.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.184+0000 7fba9feaf640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fba980959a0 0x7fba98094d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:28.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.184+0000 7fba9feaf640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fba980977d0 0x7fba9808dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:28.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.184+0000 7fba9feaf640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba981350d0 0x7fba9808e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:28.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.184+0000 7fba9feaf640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9808feb0 con 0x7fba981350d0 2024-09-13T17:44:28.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.184+0000 7fba9feaf640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba980908f0 con 0x7fba980977d0 2024-09-13T17:44:28.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.184+0000 7fba9feaf640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fba9808f970 con 0x7fba980959a0 2024-09-13T17:44:28.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.184+0000 7fba9dc4b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fba980959a0 0x7fba98094d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:28.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.184+0000 7fba9d44a640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fba980977d0 0x7fba9808dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:28.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.184+0000 7fba9e44c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba981350d0 0x7fba9808e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:28.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.184+0000 7fba9dc4b640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fba980959a0 0x7fba98094d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:42282/0 (socket says 172.21.15.102:42282) 2024-09-13T17:44:28.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.184+0000 7fba9dc4b640 1 -- 172.21.15.102:0/2594223574 learned_addr learned my addr 172.21.15.102:0/2594223574 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:28.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.185+0000 7fba9dc4b640 1 -- 172.21.15.102:0/2594223574 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fba980977d0 msgr2=0x7fba9808dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:28.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.185+0000 7fba9dc4b640 1 --2- 172.21.15.102:0/2594223574 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fba980977d0 0x7fba9808dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:28.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.185+0000 7fba9dc4b640 1 -- 172.21.15.102:0/2594223574 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba981350d0 msgr2=0x7fba9808e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:28.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.185+0000 7fba9dc4b640 1 --2- 172.21.15.102:0/2594223574 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba981350d0 0x7fba9808e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:28.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.185+0000 7fba9dc4b640 1 -- 172.21.15.102:0/2594223574 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba8c009390 con 0x7fba980959a0 2024-09-13T17:44:28.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.185+0000 7fba9e44c640 1 --2- 172.21.15.102:0/2594223574 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba981350d0 0x7fba9808e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:28.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.185+0000 7fba9d44a640 1 --2- 172.21.15.102:0/2594223574 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fba980977d0 0x7fba9808dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:28.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.185+0000 7fba9dc4b640 1 --2- 172.21.15.102:0/2594223574 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fba980959a0 0x7fba98094d10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00ec80 tx=0x7fba8c0090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:28.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.185+0000 7fba89ffb640 1 -- 172.21.15.102:0/2594223574 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c00eec0 con 0x7fba980959a0 2024-09-13T17:44:28.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.185+0000 7fba89ffb640 1 -- 172.21.15.102:0/2594223574 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba8c00cbc0 con 0x7fba980959a0 2024-09-13T17:44:28.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.185+0000 7fba9feaf640 1 -- 172.21.15.102:0/2594223574 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba90009a20 con 0x7fba980959a0 2024-09-13T17:44:28.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.185+0000 7fba89ffb640 1 -- 172.21.15.102:0/2594223574 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c013470 con 0x7fba980959a0 2024-09-13T17:44:28.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.185+0000 7fba9feaf640 1 -- 172.21.15.102:0/2594223574 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba98091650 con 0x7fba980959a0 2024-09-13T17:44:28.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.186+0000 7fba89ffb640 1 -- 172.21.15.102:0/2594223574 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fba8c0137e0 con 0x7fba980959a0 2024-09-13T17:44:28.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.186+0000 7fba9feaf640 1 -- 172.21.15.102:0/2594223574 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fba980952b0 con 0x7fba980959a0 2024-09-13T17:44:28.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.186+0000 7fba89ffb640 1 -- 172.21.15.102:0/2594223574 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 2851+0+0 (secure 0 0 0) 0x7fba8c0420f0 con 0x7fba980959a0 2024-09-13T17:44:28.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.186+0000 7fba89ffb640 1 -- 172.21.15.102:0/2594223574 --> v1:172.21.15.102:6812/35352 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fba7c03b5b0 con 0x7fba7c037910 2024-09-13T17:44:28.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.186+0000 7fba89ffb640 1 -- 172.21.15.102:0/2594223574 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba8c0123d0 con 0x7fba980959a0 2024-09-13T17:44:28.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.187+0000 7fba89ffb640 1 -- 172.21.15.102:0/2594223574 <== osd.3 v1:172.21.15.102:6812/35352 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7fba7c03b5b0 con 0x7fba7c037910 2024-09-13T17:44:28.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.198+0000 7fba9feaf640 1 -- 172.21.15.102:0/2594223574 --> v1:172.21.15.102:6812/35352 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fba98064000 con 0x7fba7c037910 2024-09-13T17:44:28.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.198+0000 7fba89ffb640 1 -- 172.21.15.102:0/2594223574 <== osd.3 v1:172.21.15.102:6812/35352 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (unknown 0 0 119112422) 0x7fba98064000 con 0x7fba7c037910 2024-09-13T17:44:28.200 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-13T17:44:28.201 INFO:teuthology.orchestra.run.smithi102.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T17:44:28.201 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-13T17:44:28.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.199+0000 7fba9feaf640 1 -- 172.21.15.102:0/2594223574 >> v1:172.21.15.102:6812/35352 conn(0x7fba7c037910 legacy=0x7fba7c039d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:28.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.199+0000 7fba9feaf640 1 -- 172.21.15.102:0/2594223574 >> v1:172.21.15.102:6816/34937 conn(0x7fba7c031160 legacy=0x7fba7c033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:28.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.199+0000 7fba9feaf640 1 -- 172.21.15.102:0/2594223574 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fba980959a0 msgr2=0x7fba98094d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:28.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.199+0000 7fba9feaf640 1 --2- 172.21.15.102:0/2594223574 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fba980959a0 0x7fba98094d10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00ec80 tx=0x7fba8c0090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:28.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.199+0000 7fba9e44c640 1 -- 172.21.15.102:0/2594223574 reap_dead start 2024-09-13T17:44:28.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.199+0000 7fba9feaf640 1 -- 172.21.15.102:0/2594223574 shutdown_connections 2024-09-13T17:44:28.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.199+0000 7fba9feaf640 1 -- 172.21.15.102:0/2594223574 >> 172.21.15.102:0/2594223574 conn(0x7fba9809f930 msgr2=0x7fba9809dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:28.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.199+0000 7fba9feaf640 1 -- 172.21.15.102:0/2594223574 shutdown_connections 2024-09-13T17:44:28.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.199+0000 7fba9feaf640 1 -- 172.21.15.102:0/2594223574 wait complete. 2024-09-13T17:44:28.210 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-13T17:44:28.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.303+0000 7fd03d19d640 1 Processor -- start 2024-09-13T17:44:28.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.303+0000 7fd03d19d640 1 -- start start 2024-09-13T17:44:28.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.303+0000 7fd03d19d640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd038128e10 0x7fd038130b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:28.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.303+0000 7fd03d19d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd0381310b0 0x7fd038133490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:28.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.303+0000 7fd03d19d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd038097be0 con 0x7fd038128330 2024-09-13T17:44:28.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.303+0000 7fd03d19d640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd038129200 con 0x7fd038128e10 2024-09-13T17:44:28.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.303+0000 7fd03d19d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd038098e70 con 0x7fd0381310b0 2024-09-13T17:44:28.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.303+0000 7fd02ffff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7fd038128e10 0x7fd038130b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:28.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.303+0000 7fd03759e640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd0381310b0 0x7fd038133490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:28.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.303+0000 7fd036d9d640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fd038128330 0x7fd038128700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:35954/0 (socket says 172.21.15.102:35954) 2024-09-13T17:44:28.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.303+0000 7fd036d9d640 1 -- 172.21.15.102:0/463580160 learned_addr learned my addr 172.21.15.102:0/463580160 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:44:28.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.304+0000 7fd03759e640 1 -- 172.21.15.102:0/463580160 >> v2:172.21.15.102:3300/0 conn(0x7fd038128e10 msgr2=0x7fd038130b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:28.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.304+0000 7fd03759e640 1 --2- 172.21.15.102:0/463580160 >> v2:172.21.15.102:3300/0 conn(0x7fd038128e10 0x7fd038130b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:28.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.304+0000 7fd03759e640 1 -- 172.21.15.102:0/463580160 >> v1:172.21.15.102:6789/0 conn(0x7fd038128330 legacy=0x7fd038128700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:28.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.304+0000 7fd03659c640 1 -- 172.21.15.102:0/463580160 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3385551305 0 0) 0x7fd038097be0 con 0x7fd038128330 2024-09-13T17:44:28.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.304+0000 7fd03759e640 1 -- 172.21.15.102:0/463580160 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd038098310 con 0x7fd0381310b0 2024-09-13T17:44:28.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.304+0000 7fd02ffff640 1 --2- 172.21.15.102:0/463580160 >> v2:172.21.15.102:3300/0 conn(0x7fd038128e10 0x7fd038130b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:28.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.304+0000 7fd03759e640 1 --2- 172.21.15.102:0/463580160 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd0381310b0 0x7fd038133490 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd03000afa0 tx=0x7fd03000a890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa69fc981ee8e095 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:28.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.305+0000 7fd03659c640 1 -- 172.21.15.102:0/463580160 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03000a2e0 con 0x7fd0381310b0 2024-09-13T17:44:28.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.305+0000 7fd03659c640 1 -- 172.21.15.102:0/463580160 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03002f050 con 0x7fd0381310b0 2024-09-13T17:44:28.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.305+0000 7fd03659c640 1 -- 172.21.15.102:0/463580160 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0300086a0 con 0x7fd0381310b0 2024-09-13T17:44:28.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.305+0000 7fd03d19d640 1 -- 172.21.15.102:0/463580160 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd0381310b0 msgr2=0x7fd038133490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:28.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.305+0000 7fd03d19d640 1 --2- 172.21.15.102:0/463580160 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd0381310b0 0x7fd038133490 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd03000afa0 tx=0x7fd03000a890 comp rx=0 tx=0).stop 2024-09-13T17:44:28.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.305+0000 7fd03d19d640 1 -- 172.21.15.102:0/463580160 shutdown_connections 2024-09-13T17:44:28.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.305+0000 7fd03d19d640 1 --2- 172.21.15.102:0/463580160 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd0381310b0 0x7fd038133490 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:28.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.305+0000 7fd03d19d640 1 --2- 172.21.15.102:0/463580160 >> v2:172.21.15.102:3300/0 conn(0x7fd038128e10 0x7fd038130b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:28.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.305+0000 7fd03d19d640 1 -- 172.21.15.102:0/463580160 >> 172.21.15.102:0/463580160 conn(0x7fd03809e820 msgr2=0x7fd03809ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:28.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.305+0000 7fd03d19d640 1 -- 172.21.15.102:0/463580160 shutdown_connections 2024-09-13T17:44:28.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.305+0000 7fd03d19d640 1 -- 172.21.15.102:0/463580160 wait complete. 2024-09-13T17:44:28.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.306+0000 7fd03d19d640 1 Processor -- start 2024-09-13T17:44:28.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.306+0000 7fd03d19d640 1 -- start start 2024-09-13T17:44:28.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.306+0000 7fd03d19d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd038128330 0x7fd038069ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:28.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.306+0000 7fd03d19d640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd038128e10 0x7fd03806a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:28.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.306+0000 7fd03d19d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0381310b0 0x7fd03806a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:28.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.306+0000 7fd03d19d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd038130e40 con 0x7fd0381310b0 2024-09-13T17:44:28.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.306+0000 7fd03d19d640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd03806c610 con 0x7fd038128330 2024-09-13T17:44:28.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.306+0000 7fd03d19d640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd03806b4d0 con 0x7fd038128e10 2024-09-13T17:44:28.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.306+0000 7fd036d9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd038128330 0x7fd038069ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:28.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.306+0000 7fd02ffff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd038128e10 0x7fd03806a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:28.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.306+0000 7fd036d9d640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd038128330 0x7fd038069ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48900/0 (socket says 172.21.15.102:48900) 2024-09-13T17:44:28.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.306+0000 7fd036d9d640 1 -- 172.21.15.102:0/2919669721 learned_addr learned my addr 172.21.15.102:0/2919669721 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:28.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.306+0000 7fd03759e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0381310b0 0x7fd03806a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:28.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.307+0000 7fd036d9d640 1 -- 172.21.15.102:0/2919669721 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd038128e10 msgr2=0x7fd03806a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:28.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.307+0000 7fd036d9d640 1 --2- 172.21.15.102:0/2919669721 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd038128e10 0x7fd03806a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:28.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.307+0000 7fd036d9d640 1 -- 172.21.15.102:0/2919669721 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0381310b0 msgr2=0x7fd03806a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:28.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.307+0000 7fd036d9d640 1 --2- 172.21.15.102:0/2919669721 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0381310b0 0x7fd03806a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:28.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.307+0000 7fd036d9d640 1 -- 172.21.15.102:0/2919669721 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd020009390 con 0x7fd038128330 2024-09-13T17:44:28.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.307+0000 7fd03759e640 1 --2- 172.21.15.102:0/2919669721 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0381310b0 0x7fd03806a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:28.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.307+0000 7fd036d9d640 1 --2- 172.21.15.102:0/2919669721 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd038128330 0x7fd038069ec0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd02800c5e0 tx=0x7fd028007d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:28.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.307+0000 7fd02effd640 1 -- 172.21.15.102:0/2919669721 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd028005aa0 con 0x7fd038128330 2024-09-13T17:44:28.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.307+0000 7fd02effd640 1 -- 172.21.15.102:0/2919669721 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd028005c00 con 0x7fd038128330 2024-09-13T17:44:28.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.307+0000 7fd02effd640 1 -- 172.21.15.102:0/2919669721 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0280070f0 con 0x7fd038128330 2024-09-13T17:44:28.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.307+0000 7fd03d19d640 1 -- 172.21.15.102:0/2919669721 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd03000ab30 con 0x7fd038128330 2024-09-13T17:44:28.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.307+0000 7fd03d19d640 1 -- 172.21.15.102:0/2919669721 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd03806ecf0 con 0x7fd038128330 2024-09-13T17:44:28.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.308+0000 7fd02effd640 1 -- 172.21.15.102:0/2919669721 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fd028007520 con 0x7fd038128330 2024-09-13T17:44:28.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.308+0000 7fd02effd640 1 -- 172.21.15.102:0/2919669721 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 2851+0+0 (secure 0 0 0) 0x7fd0280417d0 con 0x7fd038128330 2024-09-13T17:44:28.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.308+0000 7fd03d19d640 1 -- 172.21.15.102:0/2919669721 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd038064000 con 0x7fd038128330 2024-09-13T17:44:28.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.313+0000 7fd02effd640 1 -- 172.21.15.102:0/2919669721 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fd02801b400 con 0x7fd038128330 2024-09-13T17:44:28.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.465+0000 7fd03d19d640 1 -- 172.21.15.102:0/2919669721 --> v1:172.21.15.102:6816/34937 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fd03806c750 con 0x7fd018031160 2024-09-13T17:44:28.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.465+0000 7fd02effd640 1 -- 172.21.15.102:0/2919669721 <== mgr.4101 v1:172.21.15.102:6816/34937 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (unknown 3526570807 0 0) 0x7fd0200099c0 con 0x7fd018031160 2024-09-13T17:44:28.467 INFO:teuthology.orchestra.run.smithi102.stderr:instructed osd(s) 3 to deep-scrub 2024-09-13T17:44:28.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.467+0000 7fd03d19d640 1 -- 172.21.15.102:0/2919669721 >> v1:172.21.15.102:6816/34937 conn(0x7fd018031160 legacy=0x7fd018033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:28.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.467+0000 7fd03d19d640 1 -- 172.21.15.102:0/2919669721 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd038128330 msgr2=0x7fd038069ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:28.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.467+0000 7fd03d19d640 1 --2- 172.21.15.102:0/2919669721 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd038128330 0x7fd038069ec0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd02800c5e0 tx=0x7fd028007d00 comp rx=0 tx=0).stop 2024-09-13T17:44:28.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.467+0000 7fd03d19d640 1 -- 172.21.15.102:0/2919669721 shutdown_connections 2024-09-13T17:44:28.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.467+0000 7fd03d19d640 1 --2- 172.21.15.102:0/2919669721 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0381310b0 0x7fd03806a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:28.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.467+0000 7fd03d19d640 1 --2- 172.21.15.102:0/2919669721 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fd038128e10 0x7fd03806a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:28.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.467+0000 7fd03d19d640 1 --2- 172.21.15.102:0/2919669721 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fd038128330 0x7fd038069ec0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:28.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.468+0000 7fd03d19d640 1 -- 172.21.15.102:0/2919669721 >> 172.21.15.102:0/2919669721 conn(0x7fd03809e820 msgr2=0x7fd038129740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:28.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.468+0000 7fd03d19d640 1 -- 172.21.15.102:0/2919669721 shutdown_connections 2024-09-13T17:44:28.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.468+0000 7fd03d19d640 1 -- 172.21.15.102:0/2919669721 wait complete. 2024-09-13T17:44:28.480 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:44:28.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.572+0000 7fca83900640 1 Processor -- start 2024-09-13T17:44:28.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.572+0000 7fca83900640 1 -- start start 2024-09-13T17:44:28.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.572+0000 7fca83900640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fca7c06b140 0x7fca7c0998c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:28.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.572+0000 7fca83900640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fca7c09ac20 0x7fca7c099e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:28.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.572+0000 7fca83900640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca7c06ee90 con 0x7fca7c09a340 2024-09-13T17:44:28.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.572+0000 7fca83900640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c06a680 con 0x7fca7c09ac20 2024-09-13T17:44:28.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.572+0000 7fca83900640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c06ac90 con 0x7fca7c06b140 2024-09-13T17:44:28.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.572+0000 7fca80e9b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fca7c09ac20 0x7fca7c099e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:28.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.572+0000 7fca8169c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fca7c06b140 0x7fca7c0998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:28.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.573+0000 7fca81e9d640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fca7c09a340 0x7fca7c12d280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:35960/0 (socket says 172.21.15.102:35960) 2024-09-13T17:44:28.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.573+0000 7fca81e9d640 1 -- 172.21.15.102:0/3157370628 learned_addr learned my addr 172.21.15.102:0/3157370628 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:44:28.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.573+0000 7fca80e9b640 1 -- 172.21.15.102:0/3157370628 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fca7c06b140 msgr2=0x7fca7c0998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:28.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.573+0000 7fca80e9b640 1 --2- 172.21.15.102:0/3157370628 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fca7c06b140 0x7fca7c0998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:28.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.573+0000 7fca80e9b640 1 -- 172.21.15.102:0/3157370628 >> v1:172.21.15.102:6789/0 conn(0x7fca7c09a340 legacy=0x7fca7c12d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:28.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.573+0000 7fca80e9b640 1 -- 172.21.15.102:0/3157370628 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca7c099510 con 0x7fca7c09ac20 2024-09-13T17:44:28.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.573+0000 7fca8169c640 1 --2- 172.21.15.102:0/3157370628 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fca7c06b140 0x7fca7c0998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:28.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.574+0000 7fca80e9b640 1 --2- 172.21.15.102:0/3157370628 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fca7c09ac20 0x7fca7c099e00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fca6c009880 tx=0x7fca6c0090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81d04c9b1d6c8b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:28.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.574+0000 7fca73fff640 1 -- 172.21.15.102:0/3157370628 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6c02ac50 con 0x7fca7c09ac20 2024-09-13T17:44:28.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.574+0000 7fca73fff640 1 -- 172.21.15.102:0/3157370628 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca6c02adb0 con 0x7fca7c09ac20 2024-09-13T17:44:28.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.574+0000 7fca73fff640 1 -- 172.21.15.102:0/3157370628 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6c032670 con 0x7fca7c09ac20 2024-09-13T17:44:28.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.574+0000 7fca83900640 1 -- 172.21.15.102:0/3157370628 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fca7c09ac20 msgr2=0x7fca7c099e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:28.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.574+0000 7fca83900640 1 --2- 172.21.15.102:0/3157370628 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fca7c09ac20 0x7fca7c099e00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fca6c009880 tx=0x7fca6c0090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:28.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.574+0000 7fca83900640 1 -- 172.21.15.102:0/3157370628 shutdown_connections 2024-09-13T17:44:28.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.574+0000 7fca83900640 1 --2- 172.21.15.102:0/3157370628 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fca7c09ac20 0x7fca7c099e00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:28.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.574+0000 7fca83900640 1 --2- 172.21.15.102:0/3157370628 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fca7c06b140 0x7fca7c0998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:28.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.574+0000 7fca83900640 1 -- 172.21.15.102:0/3157370628 >> 172.21.15.102:0/3157370628 conn(0x7fca7c09e820 msgr2=0x7fca7c09ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:28.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.574+0000 7fca83900640 1 -- 172.21.15.102:0/3157370628 shutdown_connections 2024-09-13T17:44:28.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.574+0000 7fca83900640 1 -- 172.21.15.102:0/3157370628 wait complete. 2024-09-13T17:44:28.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.575+0000 7fca83900640 1 Processor -- start 2024-09-13T17:44:28.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.575+0000 7fca83900640 1 -- start start 2024-09-13T17:44:28.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.576+0000 7fca83900640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fca7c06b140 0x7fca7c11c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:28.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.576+0000 7fca83900640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fca7c09a340 0x7fca7c11c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:28.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.576+0000 7fca83900640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fca7c09ac20 0x7fca7c11cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:28.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.576+0000 7fca83900640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c069dd0 con 0x7fca7c09ac20 2024-09-13T17:44:28.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.576+0000 7fca83900640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c11f6d0 con 0x7fca7c09a340 2024-09-13T17:44:28.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.576+0000 7fca83900640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c11fc70 con 0x7fca7c06b140 2024-09-13T17:44:28.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.576+0000 7fca80e9b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fca7c09a340 0x7fca7c11c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:28.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.576+0000 7fca8169c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fca7c06b140 0x7fca7c11c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:28.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.576+0000 7fca8169c640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fca7c06b140 0x7fca7c11c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:42308/0 (socket says 172.21.15.102:42308) 2024-09-13T17:44:28.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.576+0000 7fca80e9b640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fca7c09a340 0x7fca7c11c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:48920/0 (socket says 172.21.15.102:48920) 2024-09-13T17:44:28.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.576+0000 7fca81e9d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fca7c09ac20 0x7fca7c11cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:28.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.576+0000 7fca8169c640 1 -- 172.21.15.102:0/317534997 learned_addr learned my addr 172.21.15.102:0/317534997 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:28.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.576+0000 7fca8169c640 1 -- 172.21.15.102:0/317534997 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fca7c09a340 msgr2=0x7fca7c11c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:28.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.576+0000 7fca8169c640 1 --2- 172.21.15.102:0/317534997 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fca7c09a340 0x7fca7c11c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:28.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.576+0000 7fca8169c640 1 -- 172.21.15.102:0/317534997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fca7c09ac20 msgr2=0x7fca7c11cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:28.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.576+0000 7fca8169c640 1 --2- 172.21.15.102:0/317534997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fca7c09ac20 0x7fca7c11cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:28.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.576+0000 7fca8169c640 1 -- 172.21.15.102:0/317534997 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca78009390 con 0x7fca7c06b140 2024-09-13T17:44:28.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.576+0000 7fca80e9b640 1 --2- 172.21.15.102:0/317534997 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fca7c09a340 0x7fca7c11c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:28.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.576+0000 7fca8169c640 1 --2- 172.21.15.102:0/317534997 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fca7c06b140 0x7fca7c11c2e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fca7800ea70 tx=0x7fca780090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:28.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.576+0000 7fca717fa640 1 -- 172.21.15.102:0/317534997 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7800ca10 con 0x7fca7c06b140 2024-09-13T17:44:28.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.576+0000 7fca717fa640 1 -- 172.21.15.102:0/317534997 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca7800cb70 con 0x7fca7c06b140 2024-09-13T17:44:28.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.576+0000 7fca83900640 1 -- 172.21.15.102:0/317534997 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca6c009390 con 0x7fca7c06b140 2024-09-13T17:44:28.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.576+0000 7fca717fa640 1 -- 172.21.15.102:0/317534997 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca78013470 con 0x7fca7c06b140 2024-09-13T17:44:28.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.576+0000 7fca83900640 1 -- 172.21.15.102:0/317534997 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca7c1203c0 con 0x7fca7c06b140 2024-09-13T17:44:28.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.577+0000 7fca717fa640 1 -- 172.21.15.102:0/317534997 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7fca78013820 con 0x7fca7c06b140 2024-09-13T17:44:28.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.577+0000 7fca717fa640 1 -- 172.21.15.102:0/317534997 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 2851+0+0 (secure 0 0 0) 0x7fca780421d0 con 0x7fca7c06b140 2024-09-13T17:44:28.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.577+0000 7fca83900640 1 -- 172.21.15.102:0/317534997 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca7c064000 con 0x7fca7c06b140 2024-09-13T17:44:28.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.582+0000 7fca717fa640 1 -- 172.21.15.102:0/317534997 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7fca78021020 con 0x7fca7c06b140 2024-09-13T17:44:28.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.729+0000 7fca83900640 1 -- 172.21.15.102:0/317534997 --> v1:172.21.15.102:6816/34937 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fca7c11e610 con 0x7fca60031160 2024-09-13T17:44:28.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.731+0000 7fca717fa640 1 -- 172.21.15.102:0/317534997 <== mgr.4101 v1:172.21.15.102:6816/34937 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+39705 (unknown 2965378022 0 4143816339) 0x7fca6c000bc0 con 0x7fca60031160 2024-09-13T17:44:28.733 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:44:28.733 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-09-13T17:44:28.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.733+0000 7fca83900640 1 -- 172.21.15.102:0/317534997 >> v1:172.21.15.102:6816/34937 conn(0x7fca60031160 legacy=0x7fca60033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:28.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.733+0000 7fca83900640 1 -- 172.21.15.102:0/317534997 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fca7c06b140 msgr2=0x7fca7c11c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:28.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.733+0000 7fca83900640 1 --2- 172.21.15.102:0/317534997 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fca7c06b140 0x7fca7c11c2e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fca7800ea70 tx=0x7fca780090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:28.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.733+0000 7fca83900640 1 -- 172.21.15.102:0/317534997 shutdown_connections 2024-09-13T17:44:28.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.733+0000 7fca83900640 1 --2- 172.21.15.102:0/317534997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fca7c09ac20 0x7fca7c11cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:28.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.733+0000 7fca83900640 1 --2- 172.21.15.102:0/317534997 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7fca7c09a340 0x7fca7c11c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:28.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.733+0000 7fca83900640 1 --2- 172.21.15.102:0/317534997 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7fca7c06b140 0x7fca7c11c2e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:28.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.733+0000 7fca83900640 1 -- 172.21.15.102:0/317534997 >> 172.21.15.102:0/317534997 conn(0x7fca7c09e820 msgr2=0x7fca7c09c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:28.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.733+0000 7fca83900640 1 -- 172.21.15.102:0/317534997 shutdown_connections 2024-09-13T17:44:28.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:28.733+0000 7fca83900640 1 -- 172.21.15.102:0/317534997 wait complete. 2024-09-13T17:44:28.745 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2024-09-13T17:44:28.443488+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":159,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1191804,"kb_used_data":2372,"kb_used_omap":6,"kb_used_meta":1189369,"kb_avail":376295556,"statfs":{"total":386547056640,"available":385326649344,"internally_reserved":0,"allocated":2428928,"data_stored":1633565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6816,"internal_metadata":1217914208},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5743,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-50,"num_write_kb":-19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001237"},"pg_stats":[{"pgid":"1.0","version":"11'52","reported_seq":98,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T17:44:23.584041+0000","last_change":"2024-09-13T17:41:55.654539+0000","last_active":"2024-09-13T17:44:23.584041+0000","last_peered":"2024-09-13T17:44:23.584041+0000","last_clean":"2024-09-13T17:44:23.584041+0000","last_became_active":"2024-09-13T17:41:55.653536+0000","last_became_peered":"2024-09-13T17:41:55.653536+0000","last_unstale":"2024-09-13T17:44:23.584041+0000","last_undegraded":"2024-09-13T17:44:23.584041+0000","last_fullsized":"2024-09-13T17:44:23.584041+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:54.532965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:54.532965+0000","last_clean_scrub_stamp":"2024-09-13T17:41:54.532965+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:56.662588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":33,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T17:44:20.378161+0000","last_change":"2024-09-13T17:42:02.137356+0000","last_active":"2024-09-13T17:44:20.378161+0000","last_peered":"2024-09-13T17:44:20.378161+0000","last_clean":"2024-09-13T17:44:20.378161+0000","last_became_active":"2024-09-13T17:42:00.116534+0000","last_became_peered":"2024-09-13T17:42:00.116534+0000","last_unstale":"2024-09-13T17:44:20.378161+0000","last_undegraded":"2024-09-13T17:44:20.378161+0000","last_fullsized":"2024-09-13T17:44:20.378161+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:06.665067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000321799,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T17:44:20.377427+0000","last_change":"2024-09-13T17:42:02.137373+0000","last_active":"2024-09-13T17:44:20.377427+0000","last_peered":"2024-09-13T17:44:20.377427+0000","last_clean":"2024-09-13T17:44:20.377427+0000","last_became_active":"2024-09-13T17:42:00.116542+0000","last_became_peered":"2024-09-13T17:42:00.116542+0000","last_unstale":"2024-09-13T17:44:20.377427+0000","last_undegraded":"2024-09-13T17:44:20.377427+0000","last_fullsized":"2024-09-13T17:44:20.377427+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:53.754563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035063,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T17:44:22.216503+0000","last_change":"2024-09-13T17:42:02.615831+0000","last_active":"2024-09-13T17:44:22.216503+0000","last_peered":"2024-09-13T17:44:22.216503+0000","last_clean":"2024-09-13T17:44:22.216503+0000","last_became_active":"2024-09-13T17:42:00.116427+0000","last_became_peered":"2024-09-13T17:42:00.116427+0000","last_unstale":"2024-09-13T17:44:22.216503+0000","last_undegraded":"2024-09-13T17:44:22.216503+0000","last_fullsized":"2024-09-13T17:44:22.216503+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:29:17.848348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000228423,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T17:44:20.377947+0000","last_change":"2024-09-13T17:42:02.138081+0000","last_active":"2024-09-13T17:44:20.377947+0000","last_peered":"2024-09-13T17:44:20.377947+0000","last_clean":"2024-09-13T17:44:20.377947+0000","last_became_active":"2024-09-13T17:42:00.116465+0000","last_became_peered":"2024-09-13T17:42:00.116465+0000","last_unstale":"2024-09-13T17:44:20.377947+0000","last_undegraded":"2024-09-13T17:44:20.377947+0000","last_fullsized":"2024-09-13T17:44:20.377947+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:40:30.342531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001026573,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T17:44:23.583120+0000","last_change":"2024-09-13T17:42:02.135135+0000","last_active":"2024-09-13T17:44:23.583120+0000","last_peered":"2024-09-13T17:44:23.583120+0000","last_clean":"2024-09-13T17:44:23.583120+0000","last_became_active":"2024-09-13T17:42:00.364240+0000","last_became_peered":"2024-09-13T17:42:00.364240+0000","last_unstale":"2024-09-13T17:44:23.583120+0000","last_undegraded":"2024-09-13T17:44:23.583120+0000","last_fullsized":"2024-09-13T17:44:23.583120+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:31:29.382006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575944,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":32,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T17:44:20.378117+0000","last_change":"2024-09-13T17:42:02.138136+0000","last_active":"2024-09-13T17:44:20.378117+0000","last_peered":"2024-09-13T17:44:20.378117+0000","last_clean":"2024-09-13T17:44:20.378117+0000","last_became_active":"2024-09-13T17:42:00.364233+0000","last_became_peered":"2024-09-13T17:42:00.364233+0000","last_unstale":"2024-09-13T17:44:20.378117+0000","last_undegraded":"2024-09-13T17:44:20.378117+0000","last_fullsized":"2024-09-13T17:44:20.378117+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:54.234194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010757130000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":33,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T17:44:23.584175+0000","last_change":"2024-09-13T17:42:02.134963+0000","last_active":"2024-09-13T17:44:23.584175+0000","last_peered":"2024-09-13T17:44:23.584175+0000","last_clean":"2024-09-13T17:44:23.584175+0000","last_became_active":"2024-09-13T17:42:00.364233+0000","last_became_peered":"2024-09-13T17:42:00.364233+0000","last_unstale":"2024-09-13T17:44:23.584175+0000","last_undegraded":"2024-09-13T17:44:23.584175+0000","last_fullsized":"2024-09-13T17:44:23.584175+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:24:31.191067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393223,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":32,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T17:44:20.378537+0000","last_change":"2024-09-13T17:42:02.138362+0000","last_active":"2024-09-13T17:44:20.378537+0000","last_peered":"2024-09-13T17:44:20.378537+0000","last_clean":"2024-09-13T17:44:20.378537+0000","last_became_active":"2024-09-13T17:42:00.364214+0000","last_became_peered":"2024-09-13T17:42:00.364214+0000","last_unstale":"2024-09-13T17:44:20.378537+0000","last_undegraded":"2024-09-13T17:44:20.378537+0000","last_fullsized":"2024-09-13T17:44:20.378537+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:59.105327+0000","last_clean_scrub_stamp":"2024-09-13T17:41:59.105327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:33.976642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012776910000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705697,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":315392,"data_stored":114083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:44:00 2024","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.30399999999999999,"15min":0.30499999999999999},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.39800000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.29299999999999998,"5min":0.317,"15min":0.32100000000000001},"min":{"1min":0.223,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.45300000000000001,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.29799999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:44:00 2024","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.34899999999999998,"15min":0.35299999999999998},"min":{"1min":0.255,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.5,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.214},{"interface":"front","average":{"1min":0.30599999999999999,"5min":0.318,"15min":0.32000000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.55200000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.27600000000000002}]},{"osd":2,"last update":"Fri Sep 13 17:44:00 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.36899999999999999,"15min":0.372},"min":{"1min":0.28000000000000003,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.51100000000000001,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.318},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.33700000000000002,"15min":0.33800000000000002},"min":{"1min":0.23499999999999999,"5min":0.216,"15min":0.216},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.35099999999999998}]}]},{"osd":2,"up_from":9,"seq":38654705697,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":299008,"data_stored":110599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:43:59 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.31,"15min":0.307},"min":{"1min":0.22500000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.34599999999999997,"15min":0.34999999999999998},"min":{"1min":0.22700000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.36499999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:43:59 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35199999999999998,"15min":0.35299999999999998},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.35699999999999998,"15min":0.35799999999999998},"min":{"1min":0.24399999999999999,"5min":0.224,"15min":0.224},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.38100000000000001}]},{"osd":3,"last update":"Fri Sep 13 17:43:59 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.377,"15min":0.379},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.372,"15min":0.371},"min":{"1min":0.28599999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.34100000000000003}]}]},{"osd":1,"up_from":9,"seq":38654705700,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298248,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073592,"statfs":{"total":96636764160,"available":96331358208,"internally_reserved":0,"allocated":909312,"data_stored":704451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:43:56 2024","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.40300000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"last":0.372},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.31,"15min":0.307},"min":{"1min":0.25,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"last":0.35499999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:43:56 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.318,"15min":0.317},"min":{"1min":0.23200000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.251,"5min":0.222,"15min":0.222},"max":{"1min":0.44800000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"last":0.318}]},{"osd":3,"last update":"Fri Sep 13 17:43:56 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.34799999999999998,"15min":0.34999999999999998},"min":{"1min":0.246,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35999999999999999,"15min":0.36099999999999999},"min":{"1min":0.26800000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.46500000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.33500000000000002}]}]},{"osd":0,"up_from":9,"seq":38654705698,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298244,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073596,"statfs":{"total":96636764160,"available":96331362304,"internally_reserved":0,"allocated":905216,"data_stored":704432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:43:57 2024","interfaces":[{"interface":"back","average":{"1min":0.26700000000000002,"5min":0.29999999999999999,"15min":0.30599999999999999},"min":{"1min":0.214,"5min":0.19,"15min":0.19},"max":{"1min":0.38200000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.307,"5min":0.311,"15min":0.311},"min":{"1min":0.22900000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.42299999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"last":0.25900000000000001}]},{"osd":2,"last update":"Fri Sep 13 17:43:57 2024","interfaces":[{"interface":"back","average":{"1min":0.29799999999999999,"5min":0.318,"15min":0.32100000000000001},"min":{"1min":0.23499999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.41599999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.31,"5min":0.32400000000000001,"15min":0.32600000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.437,"5min":0.44600000000000001,"15min":0.44600000000000001},"last":0.30499999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:43:57 2024","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.33200000000000002,"15min":0.33300000000000002},"min":{"1min":0.249,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.39700000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.316,"5min":0.33200000000000002,"15min":0.33500000000000002},"min":{"1min":0.254,"5min":0.248,"15min":0.248},"max":{"1min":0.40999999999999998,"5min":0.504,"15min":0.504},"last":0.29199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:44:28.749 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-13T17:41:54.532965+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=41, tm_sec=54, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=44, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:44:28.749 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-13T17:41:59.105327+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=41, tm_sec=59, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=44, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:44:28.749 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-13T17:41:59.105327+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=41, tm_sec=59, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=44, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:44:28.749 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-13T17:41:59.105327+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=41, tm_sec=59, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=44, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:44:28.749 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-13T17:41:59.105327+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=41, tm_sec=59, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=44, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:44:28.750 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-13T17:41:59.105327+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=41, tm_sec=59, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=44, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:44:28.750 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-13T17:41:59.105327+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=41, tm_sec=59, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=44, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:44:28.750 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-13T17:41:59.105327+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=41, tm_sec=59, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=44, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:44:28.750 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-13T17:41:59.105327+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=41, tm_sec=59, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=44, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:44:28.751 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T17:44:48.753 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:44:48.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.855+0000 7f6727434640 1 Processor -- start 2024-09-13T17:44:48.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.856+0000 7f6727434640 1 -- start start 2024-09-13T17:44:48.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.856+0000 7f6727434640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6720068920 0x7f672009d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:48.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.856+0000 7f6727434640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f672009dc50 0x7f672009e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:48.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.856+0000 7f6727434640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f672012f310 0x7f672012f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:48.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.856+0000 7f6727434640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6720068cf0 con 0x7f6720068920 2024-09-13T17:44:48.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.856+0000 7f6727434640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67200695a0 con 0x7f672009dc50 2024-09-13T17:44:48.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.856+0000 7f6727434640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f672009f090 con 0x7f672012f310 2024-09-13T17:44:48.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.856+0000 7f67259d1640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f672012f310 0x7f672012f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:48.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.856+0000 7f67251d0640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6720068920 0x7f672009d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:48.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.856+0000 7f67249cf640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f672009dc50 0x7f672009e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:48.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.856+0000 7f67259d1640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f672012f310 0x7f672012f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:51030/0 (socket says 172.21.15.102:51030) 2024-09-13T17:44:48.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.856+0000 7f67249cf640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f672009dc50 0x7f672009e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:45040/0 (socket says 172.21.15.102:45040) 2024-09-13T17:44:48.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.856+0000 7f67251d0640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f6720068920 0x7f672009d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40132/0 (socket says 172.21.15.102:40132) 2024-09-13T17:44:48.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.856+0000 7f67259d1640 1 -- 172.21.15.102:0/420252135 learned_addr learned my addr 172.21.15.102:0/420252135 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:48.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.856+0000 7f67251d0640 1 -- 172.21.15.102:0/420252135 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f672012f310 msgr2=0x7f672012f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:48.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.856+0000 7f67251d0640 1 --2- 172.21.15.102:0/420252135 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f672012f310 0x7f672012f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:48.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.856+0000 7f67251d0640 1 -- 172.21.15.102:0/420252135 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f672009dc50 msgr2=0x7f672009e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:48.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.856+0000 7f67251d0640 1 --2- 172.21.15.102:0/420252135 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f672009dc50 0x7f672009e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:48.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.856+0000 7f67251d0640 1 -- 172.21.15.102:0/420252135 --> v2:172.21.15.102:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672009cce0 con 0x7f6720068920 2024-09-13T17:44:48.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.856+0000 7f67249cf640 1 --2- 172.21.15.102:0/420252135 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f672009dc50 0x7f672009e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:48.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.856+0000 7f67259d1640 1 --2- 172.21.15.102:0/420252135 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f672012f310 0x7f672012f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:48.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.857+0000 7f67251d0640 1 --2- 172.21.15.102:0/420252135 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6720068920 0x7f672009d710 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f671c009850 tx=0x7f671c0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=528bc21be0bbcb5d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:48.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.857+0000 7f6717fff640 1 -- 172.21.15.102:0/420252135 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c02ac50 con 0x7f6720068920 2024-09-13T17:44:48.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.857+0000 7f6717fff640 1 -- 172.21.15.102:0/420252135 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671c02adb0 con 0x7f6720068920 2024-09-13T17:44:48.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.857+0000 7f6717fff640 1 -- 172.21.15.102:0/420252135 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c032670 con 0x7f6720068920 2024-09-13T17:44:48.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.858+0000 7f6727434640 1 -- 172.21.15.102:0/420252135 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6720068920 msgr2=0x7f672009d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:48.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.858+0000 7f6727434640 1 --2- 172.21.15.102:0/420252135 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6720068920 0x7f672009d710 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f671c009850 tx=0x7f671c0090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:48.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.858+0000 7f6727434640 1 -- 172.21.15.102:0/420252135 shutdown_connections 2024-09-13T17:44:48.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.858+0000 7f6727434640 1 --2- 172.21.15.102:0/420252135 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f672012f310 0x7f672012f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:48.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.858+0000 7f6727434640 1 --2- 172.21.15.102:0/420252135 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f672009dc50 0x7f672009e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:48.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.858+0000 7f6727434640 1 --2- 172.21.15.102:0/420252135 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6720068920 0x7f672009d710 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:48.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.858+0000 7f6727434640 1 -- 172.21.15.102:0/420252135 >> 172.21.15.102:0/420252135 conn(0x7f672006f320 msgr2=0x7f67200a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:48.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.858+0000 7f6727434640 1 -- 172.21.15.102:0/420252135 shutdown_connections 2024-09-13T17:44:48.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.858+0000 7f6727434640 1 -- 172.21.15.102:0/420252135 wait complete. 2024-09-13T17:44:48.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.859+0000 7f6727434640 1 Processor -- start 2024-09-13T17:44:48.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.859+0000 7f6727434640 1 -- start start 2024-09-13T17:44:48.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.859+0000 7f6727434640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6720068920 0x7f672011dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:48.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.859+0000 7f6727434640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f672009dc50 0x7f672011e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:48.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.859+0000 7f6727434640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f672012f310 0x7f672011e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:48.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.859+0000 7f6727434640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672011fd00 con 0x7f672012f310 2024-09-13T17:44:48.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.859+0000 7f6727434640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f672011f7c0 con 0x7f672009dc50 2024-09-13T17:44:48.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.859+0000 7f6727434640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6720121130 con 0x7f6720068920 2024-09-13T17:44:48.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.860+0000 7f67251d0640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6720068920 0x7f672011dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:48.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.860+0000 7f67259d1640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f672012f310 0x7f672011e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:48.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.860+0000 7f67251d0640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6720068920 0x7f672011dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:51040/0 (socket says 172.21.15.102:51040) 2024-09-13T17:44:48.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.860+0000 7f67249cf640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f672009dc50 0x7f672011e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:48.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.860+0000 7f67251d0640 1 -- 172.21.15.102:0/2869454064 learned_addr learned my addr 172.21.15.102:0/2869454064 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:48.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.860+0000 7f67251d0640 1 -- 172.21.15.102:0/2869454064 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f672009dc50 msgr2=0x7f672011e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:48.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.860+0000 7f67251d0640 1 --2- 172.21.15.102:0/2869454064 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f672009dc50 0x7f672011e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:48.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.860+0000 7f67251d0640 1 -- 172.21.15.102:0/2869454064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f672012f310 msgr2=0x7f672011e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:48.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.860+0000 7f67251d0640 1 --2- 172.21.15.102:0/2869454064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f672012f310 0x7f672011e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:48.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.860+0000 7f67251d0640 1 -- 172.21.15.102:0/2869454064 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6710009390 con 0x7f6720068920 2024-09-13T17:44:48.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.860+0000 7f67249cf640 1 --2- 172.21.15.102:0/2869454064 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f672009dc50 0x7f672011e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:48.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.860+0000 7f67259d1640 1 --2- 172.21.15.102:0/2869454064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f672012f310 0x7f672011e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:48.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.860+0000 7f67251d0640 1 --2- 172.21.15.102:0/2869454064 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6720068920 0x7f672011dce0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f671c009850 tx=0x7f671c032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:48.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.860+0000 7f67157fa640 1 -- 172.21.15.102:0/2869454064 <== mon.2 v2:172.21.15.102:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c03d700 con 0x7f6720068920 2024-09-13T17:44:48.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.860+0000 7f67157fa640 1 -- 172.21.15.102:0/2869454064 <== mon.2 v2:172.21.15.102:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671c03d860 con 0x7f6720068920 2024-09-13T17:44:48.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.860+0000 7f67157fa640 1 -- 172.21.15.102:0/2869454064 <== mon.2 v2:172.21.15.102:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c03dc00 con 0x7f6720068920 2024-09-13T17:44:48.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.860+0000 7f6727434640 1 -- 172.21.15.102:0/2869454064 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6718009ae0 con 0x7f6720068920 2024-09-13T17:44:48.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.860+0000 7f6727434640 1 -- 172.21.15.102:0/2869454064 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f671c009390 con 0x7f6720068920 2024-09-13T17:44:48.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.861+0000 7f67157fa640 1 -- 172.21.15.102:0/2869454064 <== mon.2 v2:172.21.15.102:3302/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f671c039040 con 0x7f6720068920 2024-09-13T17:44:48.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.861+0000 7f67157fa640 1 -- 172.21.15.102:0/2869454064 <== mon.2 v2:172.21.15.102:3302/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 2851+0+0 (secure 0 0 0) 0x7f671c061520 con 0x7f6720068920 2024-09-13T17:44:48.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.861+0000 7f6727434640 1 -- 172.21.15.102:0/2869454064 --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f672011ffa0 con 0x7f6720068920 2024-09-13T17:44:48.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:48.866+0000 7f67157fa640 1 -- 172.21.15.102:0/2869454064 <== mon.2 v2:172.21.15.102:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f671c037070 con 0x7f6720068920 2024-09-13T17:44:49.019 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.017+0000 7f6727434640 1 -- 172.21.15.102:0/2869454064 --> v1:172.21.15.102:6816/34937 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6720064000 con 0x7f6704031160 2024-09-13T17:44:49.020 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.018+0000 7f67157fa640 1 -- 172.21.15.102:0/2869454064 <== mgr.4101 v1:172.21.15.102:6816/34937 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+39779 (unknown 2965378022 0 2080486899) 0x7f6720064000 con 0x7f6704031160 2024-09-13T17:44:49.021 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:44:49.021 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-09-13T17:44:49.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.020+0000 7f6727434640 1 -- 172.21.15.102:0/2869454064 >> v1:172.21.15.102:6816/34937 conn(0x7f6704031160 legacy=0x7f6704033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:49.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.020+0000 7f6727434640 1 -- 172.21.15.102:0/2869454064 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6720068920 msgr2=0x7f672011dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:49.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.020+0000 7f6727434640 1 --2- 172.21.15.102:0/2869454064 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6720068920 0x7f672011dce0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f671c009850 tx=0x7f671c032000 comp rx=0 tx=0).stop 2024-09-13T17:44:49.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.020+0000 7f6727434640 1 -- 172.21.15.102:0/2869454064 shutdown_connections 2024-09-13T17:44:49.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.020+0000 7f6727434640 1 --2- 172.21.15.102:0/2869454064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f672012f310 0x7f672011e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:49.022 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.020+0000 7f6727434640 1 --2- 172.21.15.102:0/2869454064 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f672009dc50 0x7f672011e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:49.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.020+0000 7f6727434640 1 --2- 172.21.15.102:0/2869454064 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f6720068920 0x7f672011dce0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:49.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.020+0000 7f6727434640 1 -- 172.21.15.102:0/2869454064 >> 172.21.15.102:0/2869454064 conn(0x7f672006f320 msgr2=0x7f672009ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:49.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.020+0000 7f6727434640 1 -- 172.21.15.102:0/2869454064 shutdown_connections 2024-09-13T17:44:49.023 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.020+0000 7f6727434640 1 -- 172.21.15.102:0/2869454064 wait complete. 2024-09-13T17:44:49.033 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-09-13T17:44:48.447088+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":59,"ondisk_log_size":59,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1191792,"kb_used_data":2384,"kb_used_omap":6,"kb_used_meta":1189369,"kb_avail":376295568,"statfs":{"total":386547056640,"available":385326661632,"internally_reserved":0,"allocated":2441216,"data_stored":1637030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6816,"internal_metadata":1217914208},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001981"},"pg_stats":[{"pgid":"1.0","version":"35'54","reported_seq":111,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-13T17:44:29.315848+0000","last_change":"2024-09-13T17:44:29.315818+0000","last_active":"2024-09-13T17:44:29.315848+0000","last_peered":"2024-09-13T17:44:29.315848+0000","last_clean":"2024-09-13T17:44:29.315848+0000","last_became_active":"2024-09-13T17:41:55.653536+0000","last_became_peered":"2024-09-13T17:41:55.653536+0000","last_unstale":"2024-09-13T17:44:29.315848+0000","last_undegraded":"2024-09-13T17:44:29.315848+0000","last_fullsized":"2024-09-13T17:44:29.315848+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'54","last_scrub_stamp":"2024-09-13T17:44:29.315744+0000","last_deep_scrub":"35'54","last_deep_scrub_stamp":"2024-09-13T17:44:29.315744+0000","last_clean_scrub_stamp":"2024-09-13T17:44:29.315744+0000","objects_scrubbed":2,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:27:18.279388+0000","scrub_duration":0.0041251279999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"35'2","reported_seq":45,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-13T17:44:34.540086+0000","last_change":"2024-09-13T17:44:34.540049+0000","last_active":"2024-09-13T17:44:34.540086+0000","last_peered":"2024-09-13T17:44:34.540086+0000","last_clean":"2024-09-13T17:44:34.540086+0000","last_became_active":"2024-09-13T17:42:00.116534+0000","last_became_peered":"2024-09-13T17:42:00.116534+0000","last_unstale":"2024-09-13T17:44:34.540086+0000","last_undegraded":"2024-09-13T17:44:34.540086+0000","last_fullsized":"2024-09-13T17:44:34.540086+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'2","last_scrub_stamp":"2024-09-13T17:44:34.539920+0000","last_deep_scrub":"35'2","last_deep_scrub_stamp":"2024-09-13T17:44:34.539920+0000","last_clean_scrub_stamp":"2024-09-13T17:44:34.539920+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:53:52.125624+0000","scrub_duration":0.0029169170000000002,"objects_trimmed":0,"snaptrim_duration":0.000321799,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":43,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-13T17:44:31.526401+0000","last_change":"2024-09-13T17:44:31.526376+0000","last_active":"2024-09-13T17:44:31.526401+0000","last_peered":"2024-09-13T17:44:31.526401+0000","last_clean":"2024-09-13T17:44:31.526401+0000","last_became_active":"2024-09-13T17:42:00.116542+0000","last_became_peered":"2024-09-13T17:42:00.116542+0000","last_unstale":"2024-09-13T17:44:31.526401+0000","last_undegraded":"2024-09-13T17:44:31.526401+0000","last_fullsized":"2024-09-13T17:44:31.526401+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:31.526278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:31.526278+0000","last_clean_scrub_stamp":"2024-09-13T17:44:31.526278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:51:32.961967+0000","scrub_duration":0.0014213229999999999,"objects_trimmed":0,"snaptrim_duration":0.00035063,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":43,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-13T17:44:29.521870+0000","last_change":"2024-09-13T17:44:29.521847+0000","last_active":"2024-09-13T17:44:29.521870+0000","last_peered":"2024-09-13T17:44:29.521870+0000","last_clean":"2024-09-13T17:44:29.521870+0000","last_became_active":"2024-09-13T17:42:00.116427+0000","last_became_peered":"2024-09-13T17:42:00.116427+0000","last_unstale":"2024-09-13T17:44:29.521870+0000","last_undegraded":"2024-09-13T17:44:29.521870+0000","last_fullsized":"2024-09-13T17:44:29.521870+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:29.521768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:29.521768+0000","last_clean_scrub_stamp":"2024-09-13T17:44:29.521768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:20.474761+0000","scrub_duration":0.001173105,"objects_trimmed":0,"snaptrim_duration":0.000228423,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"35'3","reported_seq":46,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-13T17:44:32.572873+0000","last_change":"2024-09-13T17:44:32.572835+0000","last_active":"2024-09-13T17:44:32.572873+0000","last_peered":"2024-09-13T17:44:32.572873+0000","last_clean":"2024-09-13T17:44:32.572873+0000","last_became_active":"2024-09-13T17:42:00.116465+0000","last_became_peered":"2024-09-13T17:42:00.116465+0000","last_unstale":"2024-09-13T17:44:32.572873+0000","last_undegraded":"2024-09-13T17:44:32.572873+0000","last_fullsized":"2024-09-13T17:44:32.572873+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'3","last_scrub_stamp":"2024-09-13T17:44:32.572730+0000","last_deep_scrub":"35'3","last_deep_scrub_stamp":"2024-09-13T17:44:32.572730+0000","last_clean_scrub_stamp":"2024-09-13T17:44:32.572730+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:07:51.330222+0000","scrub_duration":0.002892833,"objects_trimmed":0,"snaptrim_duration":0.001026573,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-13T17:44:33.314391+0000","last_change":"2024-09-13T17:44:33.314355+0000","last_active":"2024-09-13T17:44:33.314391+0000","last_peered":"2024-09-13T17:44:33.314391+0000","last_clean":"2024-09-13T17:44:33.314391+0000","last_became_active":"2024-09-13T17:42:00.364240+0000","last_became_peered":"2024-09-13T17:42:00.364240+0000","last_unstale":"2024-09-13T17:44:33.314391+0000","last_undegraded":"2024-09-13T17:44:33.314391+0000","last_fullsized":"2024-09-13T17:44:33.314391+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:33.314259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:33.314259+0000","last_clean_scrub_stamp":"2024-09-13T17:44:33.314259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:37:17.909546+0000","scrub_duration":0.001426549,"objects_trimmed":0,"snaptrim_duration":0.000575944,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":43,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-13T17:44:28.531707+0000","last_change":"2024-09-13T17:44:28.531659+0000","last_active":"2024-09-13T17:44:28.531707+0000","last_peered":"2024-09-13T17:44:28.531707+0000","last_clean":"2024-09-13T17:44:28.531707+0000","last_became_active":"2024-09-13T17:42:00.364233+0000","last_became_peered":"2024-09-13T17:42:00.364233+0000","last_unstale":"2024-09-13T17:44:28.531707+0000","last_undegraded":"2024-09-13T17:44:28.531707+0000","last_fullsized":"2024-09-13T17:44:28.531707+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:28.531523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:28.531523+0000","last_clean_scrub_stamp":"2024-09-13T17:44:28.531523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:25:02.028801+0000","scrub_duration":0.001104587,"objects_trimmed":0,"snaptrim_duration":0.0010757130000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":44,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-13T17:44:31.274224+0000","last_change":"2024-09-13T17:44:31.274190+0000","last_active":"2024-09-13T17:44:31.274224+0000","last_peered":"2024-09-13T17:44:31.274224+0000","last_clean":"2024-09-13T17:44:31.274224+0000","last_became_active":"2024-09-13T17:42:00.364233+0000","last_became_peered":"2024-09-13T17:42:00.364233+0000","last_unstale":"2024-09-13T17:44:31.274224+0000","last_undegraded":"2024-09-13T17:44:31.274224+0000","last_fullsized":"2024-09-13T17:44:31.274224+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:31.274078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:31.274078+0000","last_clean_scrub_stamp":"2024-09-13T17:44:31.274078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:20:08.536083+0000","scrub_duration":0.001505349,"objects_trimmed":0,"snaptrim_duration":0.000393223,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":43,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-13T17:44:36.533664+0000","last_change":"2024-09-13T17:44:36.533638+0000","last_active":"2024-09-13T17:44:36.533664+0000","last_peered":"2024-09-13T17:44:36.533664+0000","last_clean":"2024-09-13T17:44:36.533664+0000","last_became_active":"2024-09-13T17:42:00.364214+0000","last_became_peered":"2024-09-13T17:42:00.364214+0000","last_unstale":"2024-09-13T17:44:36.533664+0000","last_undegraded":"2024-09-13T17:44:36.533664+0000","last_fullsized":"2024-09-13T17:44:36.533664+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:36.533542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:36.533542+0000","last_clean_scrub_stamp":"2024-09-13T17:44:36.533542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:31:16.634394+0000","scrub_duration":0.0014873320000000001,"objects_trimmed":0,"snaptrim_duration":0.0012776910000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705701,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297660,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074180,"statfs":{"total":96636764160,"available":96331960320,"internally_reserved":0,"allocated":315392,"data_stored":114083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:44:00 2024","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.30399999999999999,"15min":0.30499999999999999},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.39800000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.29299999999999998,"5min":0.317,"15min":0.32100000000000001},"min":{"1min":0.223,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.45300000000000001,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.372}]},{"osd":1,"last update":"Fri Sep 13 17:44:00 2024","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.34899999999999998,"15min":0.35299999999999998},"min":{"1min":0.255,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.5,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.30599999999999999,"5min":0.318,"15min":0.32000000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.55200000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.29199999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:44:00 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.36899999999999999,"15min":0.372},"min":{"1min":0.28000000000000003,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.51100000000000001,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.33700000000000002,"15min":0.33800000000000002},"min":{"1min":0.23499999999999999,"5min":0.216,"15min":0.216},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.40100000000000002}]}]},{"osd":2,"up_from":9,"seq":38654705701,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297656,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074184,"statfs":{"total":96636764160,"available":96331964416,"internally_reserved":0,"allocated":311296,"data_stored":114064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:43:59 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.31,"15min":0.307},"min":{"1min":0.22500000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.34599999999999997,"15min":0.34999999999999998},"min":{"1min":0.22700000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.253}]},{"osd":1,"last update":"Fri Sep 13 17:43:59 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35199999999999998,"15min":0.35299999999999998},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.35699999999999998,"15min":0.35799999999999998},"min":{"1min":0.24399999999999999,"5min":0.224,"15min":0.224},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.26700000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:43:59 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.377,"15min":0.379},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.315},{"interface":"front","average":{"1min":0.376,"5min":0.372,"15min":0.371},"min":{"1min":0.28599999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.32900000000000001}]}]},{"osd":1,"up_from":9,"seq":38654705704,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298240,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073600,"statfs":{"total":96636764160,"available":96331366400,"internally_reserved":0,"allocated":909312,"data_stored":704451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:43:56 2024","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.40300000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.31,"15min":0.307},"min":{"1min":0.25,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"last":0.46000000000000002}]},{"osd":2,"last update":"Fri Sep 13 17:43:56 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.318,"15min":0.317},"min":{"1min":0.23200000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.251,"5min":0.222,"15min":0.222},"max":{"1min":0.44800000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"last":0.38800000000000001}]},{"osd":3,"last update":"Fri Sep 13 17:43:56 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.34799999999999998,"15min":0.34999999999999998},"min":{"1min":0.246,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35999999999999999,"15min":0.36099999999999999},"min":{"1min":0.26800000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.46500000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.44600000000000001}]}]},{"osd":0,"up_from":9,"seq":38654705702,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298236,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073604,"statfs":{"total":96636764160,"available":96331370496,"internally_reserved":0,"allocated":905216,"data_stored":704432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:43:57 2024","interfaces":[{"interface":"back","average":{"1min":0.26700000000000002,"5min":0.29999999999999999,"15min":0.30599999999999999},"min":{"1min":0.214,"5min":0.19,"15min":0.19},"max":{"1min":0.38200000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.307,"5min":0.311,"15min":0.311},"min":{"1min":0.22900000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.42299999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"last":0.46700000000000003}]},{"osd":2,"last update":"Fri Sep 13 17:43:57 2024","interfaces":[{"interface":"back","average":{"1min":0.29799999999999999,"5min":0.318,"15min":0.32100000000000001},"min":{"1min":0.23499999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.41599999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.31,"5min":0.32400000000000001,"15min":0.32600000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.437,"5min":0.44600000000000001,"15min":0.44600000000000001},"last":0.52700000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:43:57 2024","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.33200000000000002,"15min":0.33300000000000002},"min":{"1min":0.249,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.39700000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.316,"5min":0.33200000000000002,"15min":0.33500000000000002},"min":{"1min":0.254,"5min":0.248,"15min":0.248},"max":{"1min":0.40999999999999998,"5min":0.504,"15min":0.504},"last":0.51400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:44:49.034 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-13T17:44:49.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.126+0000 7f1682309640 1 Processor -- start 2024-09-13T17:44:49.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.126+0000 7f1682309640 1 -- start start 2024-09-13T17:44:49.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.126+0000 7f1682309640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f167c096a70 0x7f167c12c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:49.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.126+0000 7f1682309640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f167c12ccd0 0x7f167c098b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:49.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.126+0000 7f1682309640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f167c099540 con 0x7f167c12c900 2024-09-13T17:44:49.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.126+0000 7f1682309640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f167c096e40 con 0x7f167c096a70 2024-09-13T17:44:49.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.126+0000 7f1682309640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f167c09a8e0 con 0x7f167c12ccd0 2024-09-13T17:44:49.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.127+0000 7f167bfff640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f167c096a70 0x7f167c12c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:49.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.127+0000 7f16808a6640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f167c12ccd0 0x7f167c098b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:49.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.127+0000 7f167b7fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f167c12c900 0x7f167c098410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:52592/0 (socket says 172.21.15.102:52592) 2024-09-13T17:44:49.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.127+0000 7f167b7fe640 1 -- 172.21.15.102:0/1344924198 learned_addr learned my addr 172.21.15.102:0/1344924198 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:44:49.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.127+0000 7f16808a6640 1 -- 172.21.15.102:0/1344924198 >> v2:172.21.15.102:3300/0 conn(0x7f167c096a70 msgr2=0x7f167c12c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:49.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.127+0000 7f16808a6640 1 --2- 172.21.15.102:0/1344924198 >> v2:172.21.15.102:3300/0 conn(0x7f167c096a70 0x7f167c12c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:49.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.127+0000 7f16808a6640 1 -- 172.21.15.102:0/1344924198 >> v1:172.21.15.102:6789/0 conn(0x7f167c12c900 legacy=0x7f167c098410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:49.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.127+0000 7f16808a6640 1 -- 172.21.15.102:0/1344924198 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f167c099760 con 0x7f167c12ccd0 2024-09-13T17:44:49.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.127+0000 7f167affd640 1 -- 172.21.15.102:0/1344924198 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3078358225 0 0) 0x7f167c099540 con 0x7f167c12c900 2024-09-13T17:44:49.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.127+0000 7f167bfff640 1 --2- 172.21.15.102:0/1344924198 >> v2:172.21.15.102:3300/0 conn(0x7f167c096a70 0x7f167c12c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:49.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.128+0000 7f16808a6640 1 --2- 172.21.15.102:0/1344924198 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f167c12ccd0 0x7f167c098b20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1674009e60 tx=0x7f1674009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba5a79642a4850e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:49.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.128+0000 7f167affd640 1 -- 172.21.15.102:0/1344924198 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1674007550 con 0x7f167c12ccd0 2024-09-13T17:44:49.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.128+0000 7f167affd640 1 -- 172.21.15.102:0/1344924198 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f167402d050 con 0x7f167c12ccd0 2024-09-13T17:44:49.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.128+0000 7f167affd640 1 -- 172.21.15.102:0/1344924198 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16740078e0 con 0x7f167c12ccd0 2024-09-13T17:44:49.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.128+0000 7f1682309640 1 -- 172.21.15.102:0/1344924198 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f167c12ccd0 msgr2=0x7f167c098b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:49.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.128+0000 7f1682309640 1 --2- 172.21.15.102:0/1344924198 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f167c12ccd0 0x7f167c098b20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1674009e60 tx=0x7f1674009780 comp rx=0 tx=0).stop 2024-09-13T17:44:49.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.128+0000 7f1682309640 1 -- 172.21.15.102:0/1344924198 shutdown_connections 2024-09-13T17:44:49.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.128+0000 7f1682309640 1 --2- 172.21.15.102:0/1344924198 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f167c12ccd0 0x7f167c098b20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:49.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.128+0000 7f1682309640 1 --2- 172.21.15.102:0/1344924198 >> v2:172.21.15.102:3300/0 conn(0x7f167c096a70 0x7f167c12c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:49.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.128+0000 7f1682309640 1 -- 172.21.15.102:0/1344924198 >> 172.21.15.102:0/1344924198 conn(0x7f167c09e6b0 msgr2=0x7f167c09eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:49.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.128+0000 7f1682309640 1 -- 172.21.15.102:0/1344924198 shutdown_connections 2024-09-13T17:44:49.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.128+0000 7f1682309640 1 -- 172.21.15.102:0/1344924198 wait complete. 2024-09-13T17:44:49.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.129+0000 7f1682309640 1 Processor -- start 2024-09-13T17:44:49.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.129+0000 7f1682309640 1 -- start start 2024-09-13T17:44:49.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.130+0000 7f1682309640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f167c096a70 0x7f167c13c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:49.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.130+0000 7f1682309640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f167c12c900 0x7f167c13cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:49.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.130+0000 7f1682309640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f167c12ccd0 0x7f167c115110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:49.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.130+0000 7f1682309640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167c1165f0 con 0x7f167c12ccd0 2024-09-13T17:44:49.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.130+0000 7f1682309640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f167c09a030 con 0x7f167c12c900 2024-09-13T17:44:49.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.130+0000 7f1682309640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f167c09d660 con 0x7f167c096a70 2024-09-13T17:44:49.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.130+0000 7f167bfff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f167c096a70 0x7f167c13c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:49.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.130+0000 7f167b7fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f167c12c900 0x7f167c13cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:49.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.130+0000 7f16808a6640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f167c12ccd0 0x7f167c115110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:49.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.130+0000 7f167b7fe640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f167c12c900 0x7f167c13cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:45060/0 (socket says 172.21.15.102:45060) 2024-09-13T17:44:49.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.130+0000 7f167bfff640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f167c096a70 0x7f167c13c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3302/0 says I am v2:172.21.15.102:51056/0 (socket says 172.21.15.102:51056) 2024-09-13T17:44:49.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.130+0000 7f167b7fe640 1 -- 172.21.15.102:0/862501257 learned_addr learned my addr 172.21.15.102:0/862501257 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:49.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.130+0000 7f16808a6640 1 -- 172.21.15.102:0/862501257 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f167c096a70 msgr2=0x7f167c13c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:49.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.130+0000 7f16808a6640 1 --2- 172.21.15.102:0/862501257 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f167c096a70 0x7f167c13c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:49.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.130+0000 7f16808a6640 1 -- 172.21.15.102:0/862501257 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f167c12c900 msgr2=0x7f167c13cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:49.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.130+0000 7f16808a6640 1 --2- 172.21.15.102:0/862501257 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f167c12c900 0x7f167c13cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:49.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.130+0000 7f16808a6640 1 -- 172.21.15.102:0/862501257 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166c009390 con 0x7f167c12ccd0 2024-09-13T17:44:49.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.130+0000 7f167b7fe640 1 --2- 172.21.15.102:0/862501257 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f167c12c900 0x7f167c13cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:49.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.130+0000 7f16808a6640 1 --2- 172.21.15.102:0/862501257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f167c12ccd0 0x7f167c115110 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1674031000 tx=0x7f1674037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:49.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.130+0000 7f1663fff640 1 -- 172.21.15.102:0/862501257 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1674031a00 con 0x7f167c12ccd0 2024-09-13T17:44:49.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.130+0000 7f1663fff640 1 -- 172.21.15.102:0/862501257 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1674040070 con 0x7f167c12ccd0 2024-09-13T17:44:49.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.130+0000 7f1682309640 1 -- 172.21.15.102:0/862501257 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1674009a20 con 0x7f167c12ccd0 2024-09-13T17:44:49.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.130+0000 7f1663fff640 1 -- 172.21.15.102:0/862501257 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1674031dd0 con 0x7f167c12ccd0 2024-09-13T17:44:49.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.131+0000 7f1682309640 1 -- 172.21.15.102:0/862501257 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f167c1184c0 con 0x7f167c12ccd0 2024-09-13T17:44:49.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.131+0000 7f1663fff640 1 -- 172.21.15.102:0/862501257 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f1674042030 con 0x7f167c12ccd0 2024-09-13T17:44:49.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.131+0000 7f1663fff640 1 -- 172.21.15.102:0/862501257 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 2851+0+0 (secure 0 0 0) 0x7f1674063410 con 0x7f167c12ccd0 2024-09-13T17:44:49.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.132+0000 7f1682309640 1 -- 172.21.15.102:0/862501257 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f167c064000 con 0x7f167c12ccd0 2024-09-13T17:44:49.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.136+0000 7f1663fff640 1 -- 172.21.15.102:0/862501257 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f167403c5e0 con 0x7f167c12ccd0 2024-09-13T17:44:49.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.289+0000 7f1682309640 1 -- 172.21.15.102:0/862501257 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f167c129c00 con 0x7f167c12ccd0 2024-09-13T17:44:49.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.291+0000 7f1663fff640 1 -- 172.21.15.102:0/862501257 <== mon.0 v2:172.21.15.102:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f167403c5e0 con 0x7f167c12ccd0 2024-09-13T17:44:49.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.291+0000 7f1663fff640 1 -- 172.21.15.102:0/862501257 <== mon.0 v2:172.21.15.102:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v8)=0 v8) v1 ==== 123+0+0 (secure 0 0 0) 0x7f16740443e0 con 0x7f167c12ccd0 2024-09-13T17:44:49.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.294+0000 7f1682309640 1 -- 172.21.15.102:0/862501257 >> v1:172.21.15.102:6816/34937 conn(0x7f1668031160 legacy=0x7f1668033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:49.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.294+0000 7f1682309640 1 -- 172.21.15.102:0/862501257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f167c12ccd0 msgr2=0x7f167c115110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:49.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.294+0000 7f1682309640 1 --2- 172.21.15.102:0/862501257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f167c12ccd0 0x7f167c115110 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1674031000 tx=0x7f1674037000 comp rx=0 tx=0).stop 2024-09-13T17:44:49.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.294+0000 7f1682309640 1 -- 172.21.15.102:0/862501257 shutdown_connections 2024-09-13T17:44:49.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.294+0000 7f1682309640 1 --2- 172.21.15.102:0/862501257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f167c12ccd0 0x7f167c115110 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:49.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.294+0000 7f1682309640 1 --2- 172.21.15.102:0/862501257 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f167c12c900 0x7f167c13cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:49.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.294+0000 7f1682309640 1 --2- 172.21.15.102:0/862501257 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f167c096a70 0x7f167c13c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:49.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.294+0000 7f1682309640 1 -- 172.21.15.102:0/862501257 >> 172.21.15.102:0/862501257 conn(0x7f167c09e6b0 msgr2=0x7f167c12ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:49.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.294+0000 7f1682309640 1 -- 172.21.15.102:0/862501257 shutdown_connections 2024-09-13T17:44:49.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.294+0000 7f1682309640 1 -- 172.21.15.102:0/862501257 wait complete. 2024-09-13T17:44:49.307 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T17:44:49.307 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:44:49.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.398+0000 7f1bb8b09640 1 Processor -- start 2024-09-13T17:44:49.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.398+0000 7f1bb8b09640 1 -- start start 2024-09-13T17:44:49.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.399+0000 7f1bb8b09640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1bb4068950 0x7f1bb409d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:49.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.399+0000 7f1bb8b09640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1bb409d3e0 0x7f1bb409ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:49.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.399+0000 7f1bb8b09640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bb4069080 con 0x7f1bb412f420 2024-09-13T17:44:49.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.399+0000 7f1bb8b09640 1 -- --> v2:172.21.15.102:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bb4068d20 con 0x7f1bb409d3e0 2024-09-13T17:44:49.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.399+0000 7f1bb8b09640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb409cf30 con 0x7f1bb4068950 2024-09-13T17:44:49.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.399+0000 7f1ba9d9b640 1 --2- >> v2:172.21.15.102:3300/0 conn(0x7f1bb409d3e0 0x7f1bb409ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:49.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.399+0000 7f1bb259c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1bb4068950 0x7f1bb409d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:49.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.399+0000 7f1bb2d9d640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f1bb412f420 0x7f1bb412f860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:52594/0 (socket says 172.21.15.102:52594) 2024-09-13T17:44:49.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.399+0000 7f1bb2d9d640 1 -- 172.21.15.102:0/4067479567 learned_addr learned my addr 172.21.15.102:0/4067479567 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-13T17:44:49.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.399+0000 7f1bb259c640 1 -- 172.21.15.102:0/4067479567 >> v2:172.21.15.102:3300/0 conn(0x7f1bb409d3e0 msgr2=0x7f1bb409ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:44:49.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.399+0000 7f1bb259c640 1 --2- 172.21.15.102:0/4067479567 >> v2:172.21.15.102:3300/0 conn(0x7f1bb409d3e0 0x7f1bb409ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:49.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.399+0000 7f1bb259c640 1 -- 172.21.15.102:0/4067479567 >> v1:172.21.15.102:6789/0 conn(0x7f1bb412f420 legacy=0x7f1bb412f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:49.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.399+0000 7f1bb1d9b640 1 -- 172.21.15.102:0/4067479567 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1711158787 0 0) 0x7f1bb4069080 con 0x7f1bb412f420 2024-09-13T17:44:49.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.399+0000 7f1bb259c640 1 -- 172.21.15.102:0/4067479567 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb409f110 con 0x7f1bb4068950 2024-09-13T17:44:49.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.399+0000 7f1ba9d9b640 1 --2- 172.21.15.102:0/4067479567 >> v2:172.21.15.102:3300/0 conn(0x7f1bb409d3e0 0x7f1bb409ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:49.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.400+0000 7f1bb259c640 1 --2- 172.21.15.102:0/4067479567 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1bb4068950 0x7f1bb409d870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4009880 tx=0x7f1ba40090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfaad9c0d4d38695 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:49.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.400+0000 7f1bb1d9b640 1 -- 172.21.15.102:0/4067479567 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba402ace0 con 0x7f1bb4068950 2024-09-13T17:44:49.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.400+0000 7f1bb1d9b640 1 -- 172.21.15.102:0/4067479567 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f1ba402ae40 con 0x7f1bb4068950 2024-09-13T17:44:49.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.400+0000 7f1bb1d9b640 1 -- 172.21.15.102:0/4067479567 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4032690 con 0x7f1bb4068950 2024-09-13T17:44:49.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.400+0000 7f1bb8b09640 1 -- 172.21.15.102:0/4067479567 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1bb4068950 msgr2=0x7f1bb409d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:49.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.400+0000 7f1bb8b09640 1 --2- 172.21.15.102:0/4067479567 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1bb4068950 0x7f1bb409d870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4009880 tx=0x7f1ba40090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:49.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.401+0000 7f1bb8b09640 1 -- 172.21.15.102:0/4067479567 shutdown_connections 2024-09-13T17:44:49.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.401+0000 7f1bb8b09640 1 --2- 172.21.15.102:0/4067479567 >> v2:172.21.15.102:3300/0 conn(0x7f1bb409d3e0 0x7f1bb409ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:49.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.401+0000 7f1bb8b09640 1 --2- 172.21.15.102:0/4067479567 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1bb4068950 0x7f1bb409d870 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:49.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.401+0000 7f1bb8b09640 1 -- 172.21.15.102:0/4067479567 >> 172.21.15.102:0/4067479567 conn(0x7f1bb406f2e0 msgr2=0x7f1bb40a2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:49.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.401+0000 7f1bb8b09640 1 -- 172.21.15.102:0/4067479567 shutdown_connections 2024-09-13T17:44:49.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.401+0000 7f1bb8b09640 1 -- 172.21.15.102:0/4067479567 wait complete. 2024-09-13T17:44:49.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.401+0000 7f1bb8b09640 1 Processor -- start 2024-09-13T17:44:49.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.401+0000 7f1bb8b09640 1 -- start start 2024-09-13T17:44:49.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.402+0000 7f1bb8b09640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1bb4068950 0x7f1bb413ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:49.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.402+0000 7f1bb8b09640 1 --2- >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1bb409d3e0 0x7f1bb413f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:49.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.402+0000 7f1bb8b09640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bb412f420 0x7f1bb4119150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:49.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.402+0000 7f1bb8b09640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb409e720 con 0x7f1bb412f420 2024-09-13T17:44:49.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.402+0000 7f1bb8b09640 1 -- --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb411bcb0 con 0x7f1bb4068950 2024-09-13T17:44:49.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.402+0000 7f1bb8b09640 1 -- --> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb411a340 con 0x7f1bb409d3e0 2024-09-13T17:44:49.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.402+0000 7f1bb259c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1bb4068950 0x7f1bb413ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:49.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.402+0000 7f1bb259c640 1 --2- >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1bb4068950 0x7f1bb413ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3301/0 says I am v2:172.21.15.102:45080/0 (socket says 172.21.15.102:45080) 2024-09-13T17:44:49.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.402+0000 7f1bb259c640 1 -- 172.21.15.102:0/3698671119 learned_addr learned my addr 172.21.15.102:0/3698671119 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-13T17:44:49.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.402+0000 7f1bb2d9d640 1 --2- 172.21.15.102:0/3698671119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bb412f420 0x7f1bb4119150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:49.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.402+0000 7f1ba9d9b640 1 --2- 172.21.15.102:0/3698671119 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1bb409d3e0 0x7f1bb413f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:49.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.402+0000 7f1bb259c640 1 -- 172.21.15.102:0/3698671119 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1bb409d3e0 msgr2=0x7f1bb413f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:49.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.402+0000 7f1bb259c640 1 --2- 172.21.15.102:0/3698671119 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1bb409d3e0 0x7f1bb413f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:49.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.402+0000 7f1bb259c640 1 -- 172.21.15.102:0/3698671119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bb412f420 msgr2=0x7f1bb4119150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:49.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.402+0000 7f1bb259c640 1 --2- 172.21.15.102:0/3698671119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bb412f420 0x7f1bb4119150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:49.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.402+0000 7f1bb259c640 1 -- 172.21.15.102:0/3698671119 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b9c009390 con 0x7f1bb4068950 2024-09-13T17:44:49.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.402+0000 7f1bb2d9d640 1 --2- 172.21.15.102:0/3698671119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bb412f420 0x7f1bb4119150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:49.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.402+0000 7f1ba9d9b640 1 --2- 172.21.15.102:0/3698671119 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1bb409d3e0 0x7f1bb413f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:44:49.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.402+0000 7f1bb259c640 1 --2- 172.21.15.102:0/3698671119 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1bb4068950 0x7f1bb413ed10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4009880 tx=0x7f1ba40090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:49.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.403+0000 7f1baaffd640 1 -- 172.21.15.102:0/3698671119 <== mon.1 v2:172.21.15.102:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba403b610 con 0x7f1bb4068950 2024-09-13T17:44:49.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.403+0000 7f1baaffd640 1 -- 172.21.15.102:0/3698671119 <== mon.1 v2:172.21.15.102:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f1ba403b770 con 0x7f1bb4068950 2024-09-13T17:44:49.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.403+0000 7f1baaffd640 1 -- 172.21.15.102:0/3698671119 <== mon.1 v2:172.21.15.102:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba403bb70 con 0x7f1bb4068950 2024-09-13T17:44:49.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.403+0000 7f1bb8b09640 1 -- 172.21.15.102:0/3698671119 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ba4009390 con 0x7f1bb4068950 2024-09-13T17:44:49.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.403+0000 7f1bb8b09640 1 -- 172.21.15.102:0/3698671119 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb411ab30 con 0x7f1bb4068950 2024-09-13T17:44:49.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.403+0000 7f1baaffd640 1 -- 172.21.15.102:0/3698671119 <== mon.1 v2:172.21.15.102:3301/0 4 ==== mgrmap(e 5) v1 ==== 36646+0+0 (secure 0 0 0) 0x7f1ba4039040 con 0x7f1bb4068950 2024-09-13T17:44:49.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.404+0000 7f1baaffd640 1 -- 172.21.15.102:0/3698671119 <== mon.1 v2:172.21.15.102:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 2851+0+0 (secure 0 0 0) 0x7f1ba4061eb0 con 0x7f1bb4068950 2024-09-13T17:44:49.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.404+0000 7f1bb8b09640 1 -- 172.21.15.102:0/3698671119 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bb4064000 con 0x7f1bb4068950 2024-09-13T17:44:49.410 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.408+0000 7f1baaffd640 1 -- 172.21.15.102:0/3698671119 <== mon.1 v2:172.21.15.102:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170909 (secure 0 0 0) 0x7f1ba4049070 con 0x7f1bb4068950 2024-09-13T17:44:49.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.591+0000 7f1bb8b09640 1 -- 172.21.15.102:0/3698671119 --> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1bb405e130 con 0x7f1bb4068950 2024-09-13T17:44:49.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.591+0000 7f1baaffd640 1 -- 172.21.15.102:0/3698671119 <== mon.1 v2:172.21.15.102:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+1198 (secure 0 0 0) 0x7f1ba4037030 con 0x7f1bb4068950 2024-09-13T17:44:49.594 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-13T17:44:49.594 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6817/582653324","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6817","nonce":582653324}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2024-09-13T17:44:49.594 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 13 2024-09-13T17:44:49.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.593+0000 7f1bb8b09640 1 -- 172.21.15.102:0/3698671119 >> v1:172.21.15.102:6816/34937 conn(0x7f1b94031160 legacy=0x7f1b94033610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:49.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.593+0000 7f1bb8b09640 1 -- 172.21.15.102:0/3698671119 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1bb4068950 msgr2=0x7f1bb413ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:49.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.593+0000 7f1bb8b09640 1 --2- 172.21.15.102:0/3698671119 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1bb4068950 0x7f1bb413ed10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4009880 tx=0x7f1ba40090f0 comp rx=0 tx=0).stop 2024-09-13T17:44:49.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.593+0000 7f1bb8b09640 1 -- 172.21.15.102:0/3698671119 shutdown_connections 2024-09-13T17:44:49.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.593+0000 7f1bb8b09640 1 --2- 172.21.15.102:0/3698671119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bb412f420 0x7f1bb4119150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:49.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.593+0000 7f1bb8b09640 1 --2- 172.21.15.102:0/3698671119 >> [v2:172.21.15.102:3302/0,v1:172.21.15.102:6791/0] conn(0x7f1bb409d3e0 0x7f1bb413f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:49.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.593+0000 7f1bb8b09640 1 --2- 172.21.15.102:0/3698671119 >> [v2:172.21.15.102:3301/0,v1:172.21.15.102:6790/0] conn(0x7f1bb4068950 0x7f1bb413ed10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:49.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.593+0000 7f1bb8b09640 1 -- 172.21.15.102:0/3698671119 >> 172.21.15.102:0/3698671119 conn(0x7f1bb406f2e0 msgr2=0x7f1bb4132510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:49.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.593+0000 7f1bb8b09640 1 -- 172.21.15.102:0/3698671119 shutdown_connections 2024-09-13T17:44:49.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-13T17:44:49.593+0000 7f1bb8b09640 1 -- 172.21.15.102:0/3698671119 wait complete. 2024-09-13T17:44:49.606 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-13T17:44:49.606 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-13T17:44:49.606 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T17:44:49.606 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:44:49.615 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T17:44:49.615 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-13T17:44:49.615 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-13T17:44:49.615 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:44:49.688 INFO:tasks.ceph.osd.0:Stopped 2024-09-13T17:44:49.688 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-13T17:44:49.688 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:44:49.752 INFO:tasks.ceph.osd.1:Stopped 2024-09-13T17:44:49.753 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-13T17:44:49.753 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:44:49.820 INFO:tasks.ceph.osd.2:Stopped 2024-09-13T17:44:49.820 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-13T17:44:49.821 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:44:49.888 INFO:tasks.ceph.osd.3:Stopped 2024-09-13T17:44:49.888 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-13T17:44:49.888 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-13T17:44:49.889 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:44:49.951 INFO:tasks.ceph.mgr.x:Stopped 2024-09-13T17:44:49.951 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-13T17:44:49.952 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T17:44:49.952 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:44:50.003 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T17:44:50.003 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-13T17:44:50.003 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:44:50.053 INFO:tasks.ceph.mon.b:Stopped 2024-09-13T17:44:50.054 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-13T17:44:50.054 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:44:50.104 INFO:tasks.ceph.mon.c:Stopped 2024-09-13T17:44:50.104 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-13T17:44:50.105 DEBUG:teuthology.orchestra.run.smithi102:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'overall HEALTH_' | egrep -v '\(FS_' | egrep -v '\(MDS_' | egrep -v '\(OSD_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(CACHE_POOL_' | egrep -v '\(POOL_' | egrep -v '\(MGR_DOWN\)' | egrep -v '\(PG_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v 'Monitor daemon marked osd' | egrep -v 'Behind on trimming' | egrep -v 'Manager daemon' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-13T17:44:50.105 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-09-13T17:44:50.105 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-09-13T17:44:50.179 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi102.front.sepia.ceph.com 2024-09-13T17:44:50.179 DEBUG:teuthology.orchestra.run.smithi102:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-13T17:44:50.372 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi102.front.sepia.ceph.com 2024-09-13T17:44:50.372 DEBUG:teuthology.orchestra.run.smithi102:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-13T17:44:50.459 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi102.front.sepia.ceph.com 2024-09-13T17:44:50.459 DEBUG:teuthology.orchestra.run.smithi102:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-13T17:44:50.528 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi102.front.sepia.ceph.com 2024-09-13T17:44:50.528 DEBUG:teuthology.orchestra.run.smithi102:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-13T17:44:50.626 INFO:tasks.ceph:Archiving mon data... 2024-09-13T17:44:50.627 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903920/data/mon.a.tgz 2024-09-13T17:44:50.627 DEBUG:teuthology.orchestra.run.smithi102:> mktemp 2024-09-13T17:44:50.645 INFO:teuthology.orchestra.run.smithi102.stdout:/tmp/tmp.1FFRQcd1PR 2024-09-13T17:44:50.646 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.1FFRQcd1PR 2024-09-13T17:44:50.762 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 0666 /tmp/tmp.1FFRQcd1PR 2024-09-13T17:44:50.845 DEBUG:teuthology.orchestra.remote:smithi102:/tmp/tmp.1FFRQcd1PR is 197KB 2024-09-13T17:44:50.876 DEBUG:teuthology.orchestra.run.smithi102:> rm -fr /tmp/tmp.1FFRQcd1PR 2024-09-13T17:44:50.891 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903920/data/mon.b.tgz 2024-09-13T17:44:50.892 DEBUG:teuthology.orchestra.run.smithi102:> mktemp 2024-09-13T17:44:50.946 INFO:teuthology.orchestra.run.smithi102.stdout:/tmp/tmp.7jHl3DTAdc 2024-09-13T17:44:50.946 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.7jHl3DTAdc 2024-09-13T17:44:51.062 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 0666 /tmp/tmp.7jHl3DTAdc 2024-09-13T17:44:51.145 DEBUG:teuthology.orchestra.remote:smithi102:/tmp/tmp.7jHl3DTAdc is 253KB 2024-09-13T17:44:51.239 DEBUG:teuthology.orchestra.run.smithi102:> rm -fr /tmp/tmp.7jHl3DTAdc 2024-09-13T17:44:51.254 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903920/data/mon.c.tgz 2024-09-13T17:44:51.254 DEBUG:teuthology.orchestra.run.smithi102:> mktemp 2024-09-13T17:44:51.309 INFO:teuthology.orchestra.run.smithi102.stdout:/tmp/tmp.P6mRXJjx1h 2024-09-13T17:44:51.309 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.P6mRXJjx1h 2024-09-13T17:44:51.426 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 0666 /tmp/tmp.P6mRXJjx1h 2024-09-13T17:44:51.509 DEBUG:teuthology.orchestra.remote:smithi102:/tmp/tmp.P6mRXJjx1h is 253KB 2024-09-13T17:44:51.602 DEBUG:teuthology.orchestra.run.smithi102:> rm -fr /tmp/tmp.P6mRXJjx1h 2024-09-13T17:44:51.617 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-13T17:44:51.621 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T17:44:51.659 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T17:44:51.691 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-13T17:44:51.692 DEBUG:teuthology.orchestra.run.smithi102:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T17:44:51.724 DEBUG:teuthology.orchestra.run.smithi171:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T17:44:51.748 INFO:teuthology.orchestra.run.smithi102.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T17:44:51.764 INFO:teuthology.orchestra.run.smithi171.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T17:44:51.769 INFO:tasks.ceph:Archiving crash dumps... 2024-09-13T17:44:51.771 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903920/remote/smithi102/crash 2024-09-13T17:44:51.772 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T17:44:51.817 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903920/remote/smithi171/crash 2024-09-13T17:44:51.818 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T17:44:51.941 INFO:tasks.ceph:Compressing logs... 2024-09-13T17:44:51.941 DEBUG:teuthology.orchestra.run.smithi102:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T17:44:51.944 DEBUG:teuthology.orchestra.run.smithi171:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T17:44:53.259 INFO:tasks.ceph:Archiving logs... 2024-09-13T17:44:53.259 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903920/remote/smithi102/log 2024-09-13T17:44:53.261 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T17:44:53.635 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903920/remote/smithi171/log 2024-09-13T17:44:53.636 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T17:44:53.669 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-13T17:44:53.695 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-13T17:44:53.696 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T17:44:53.698 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T17:44:53.831 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-13T17:44:53.845 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-13T17:44:53.846 DEBUG:teuthology.orchestra.run.smithi102:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T17:44:53.849 DEBUG:teuthology.orchestra.run.smithi171:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T17:44:53.861 INFO:teuthology.orchestra.run.smithi102.stderr:bash: line 1: ntpq: command not found 2024-09-13T17:44:53.865 INFO:teuthology.orchestra.run.smithi102.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T17:44:53.865 INFO:teuthology.orchestra.run.smithi102.stdout:=============================================================================== 2024-09-13T17:44:53.865 INFO:teuthology.orchestra.run.smithi102.stdout:^* hv01.front.sepia.ceph.com 3 6 377 40 -385us[ -378us] +/- 39ms 2024-09-13T17:44:53.865 INFO:teuthology.orchestra.run.smithi102.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 40 +1376us[+1376us] +/- 62ms 2024-09-13T17:44:53.865 INFO:teuthology.orchestra.run.smithi102.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 40 -1051us[-1051us] +/- 86ms 2024-09-13T17:44:53.865 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:44:53.866 INFO:teuthology.orchestra.run.smithi171.stderr:bash: line 1: ntpq: command not found 2024-09-13T17:44:53.871 INFO:teuthology.orchestra.run.smithi171.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T17:44:53.871 INFO:teuthology.orchestra.run.smithi171.stdout:=============================================================================== 2024-09-13T17:44:53.871 INFO:teuthology.orchestra.run.smithi171.stdout:^* hv01.front.sepia.ceph.com 3 6 377 38 -380us[ -372us] +/- 39ms 2024-09-13T17:44:53.871 INFO:teuthology.orchestra.run.smithi171.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 40 +1384us[+1392us] +/- 62ms 2024-09-13T17:44:53.871 INFO:teuthology.orchestra.run.smithi171.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 39 -1042us[-1034us] +/- 86ms 2024-09-13T17:44:53.871 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T17:44:53.872 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-13T17:44:53.887 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-13T17:44:53.887 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-13T17:44:53.898 DEBUG:teuthology.orchestra.run.smithi102:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-13T17:44:53.972 DEBUG:teuthology.orchestra.run.smithi171:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-13T17:44:54.041 DEBUG:teuthology.orchestra.run.smithi102:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-13T17:44:54.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:44:54.069 DEBUG:teuthology.orchestra.run.smithi171:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-13T17:44:54.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:44:54.098 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-13T17:44:54.109 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-13T17:44:54.120 INFO:teuthology.task.internal:Duration was 531.673592 seconds 2024-09-13T17:44:54.120 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-13T17:44:54.137 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-13T17:44:54.137 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T17:44:54.139 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T17:44:54.175 INFO:teuthology.orchestra.run.smithi102.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T17:44:54.180 INFO:teuthology.orchestra.run.smithi171.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T17:44:54.440 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-13T17:44:54.441 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi102.front.sepia.ceph.com 2024-09-13T17:44:54.441 DEBUG:teuthology.orchestra.run.smithi102:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T17:44:54.461 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi171.front.sepia.ceph.com 2024-09-13T17:44:54.462 DEBUG:teuthology.orchestra.run.smithi171:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T17:44:54.507 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-13T17:44:54.508 DEBUG:teuthology.orchestra.run.smithi102:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T17:44:54.510 DEBUG:teuthology.orchestra.run.smithi171:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T17:44:54.575 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-13T17:44:54.575 DEBUG:teuthology.orchestra.run.smithi102:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T17:44:54.578 DEBUG:teuthology.orchestra.run.smithi171:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T17:44:54.715 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-13T17:44:54.732 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-13T17:44:54.732 DEBUG:teuthology.orchestra.run.smithi102:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T17:44:54.756 DEBUG:teuthology.orchestra.run.smithi171:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T17:44:54.782 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-13T17:44:54.794 DEBUG:teuthology.orchestra.run.smithi102:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T17:44:54.799 DEBUG:teuthology.orchestra.run.smithi171:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T17:44:54.819 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern = core 2024-09-13T17:44:54.845 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern = core 2024-09-13T17:44:54.859 DEBUG:teuthology.orchestra.run.smithi102:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T17:44:54.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:44:54.898 DEBUG:teuthology.orchestra.run.smithi171:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T17:44:54.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:44:54.914 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-13T17:44:54.927 INFO:teuthology.task.internal:Transferring archived files... 2024-09-13T17:44:54.928 DEBUG:teuthology.misc:Transferring archived files from smithi102:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903920/remote/smithi102 2024-09-13T17:44:54.928 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T17:44:54.972 DEBUG:teuthology.misc:Transferring archived files from smithi171:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903920/remote/smithi171 2024-09-13T17:44:54.972 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T17:44:55.007 INFO:teuthology.task.internal:Removing archive directory... 2024-09-13T17:44:55.007 DEBUG:teuthology.orchestra.run.smithi102:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T17:44:55.009 DEBUG:teuthology.orchestra.run.smithi171:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T17:44:55.057 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-13T17:44:55.069 INFO:teuthology.task.internal:Not uploading archives. 2024-09-13T17:44:55.069 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-13T17:44:55.080 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-13T17:44:55.080 DEBUG:teuthology.orchestra.run.smithi102:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T17:44:55.082 DEBUG:teuthology.orchestra.run.smithi171:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T17:44:55.095 INFO:teuthology.orchestra.run.smithi102.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 17:44 /home/ubuntu/cephtest 2024-09-13T17:44:55.106 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-09-13T17:44:55.106 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-09-13T17:44:55.112 INFO:teuthology.orchestra.run.smithi171.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 17:44 /home/ubuntu/cephtest 2024-09-13T17:44:55.113 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-13T17:44:55.125 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-13T17:44:55.149 INFO:teuthology.run:Summary data: description: fs/upgrade/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/quincy 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/new_ops/{newops}}} duration: 531.6735920906067 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-09-13T17:44:55.149 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T17:44:55.241 INFO:teuthology.run:pass